Karylab-cklius/vllm
1
0
Fork 1
Code Issues Pull Requests 1 Actions Packages Projects Releases Wiki Activity
16,247 Commits 352 Branches 175 Tags
2fa1f8ec00cf85b15422cb4c0e8eb3632ee13ea8
Commit Graph
16247 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Or OzeriandGitHub 2fa1f8ec00 [kv_offload+HMA][13/N]: Enable HMA support (#41445) 
This is the final PR in a series to enables HMA support for the
offloading connector. The connector advertises `SupportsHMA`
and is validated with unit tests and e2e tests.

Signed-off-by: Or Ozeri <oro@il.ibm.com>
 2026-05-01 12:30:03 +01:00
raviguptaamdandGitHub 7075df79b3 [ROCm] Enable DBO (Dynamic Batch Optimization) on ROCm (#34726) 
Signed-off-by: raviguptaamd <ravi.gupta@amd.com>
 2026-05-01 09:18:30 +00:00
Yuyi AoandGitHub 0dbaf9daad Refractor longcat loading to use AutoWeightsLoader (#41448) 
Signed-off-by: George-ao <yuyiao772@gmail.com>
 2026-05-01 09:07:23 +00:00
Seiji EicherGitHubMarwan SarieddineClaude
 		a3ec4a35f5 [Bugfix][Metrics] Fix RayPrometheusMetric.labels() returning shared labeled child (#40840) 
When vLLM runs with Ray Prometheus `vllm:request_success{finished_reason=...}`
only ever increments the repetition bucket regardless of the request's actual finish
reason; stop, length, abort, and error stay at zero. Root cause was `labels()` mutated
the wrapped Ray metric's default tags in place and returned self, so every `.labels(...)`
call on a given wrapper returned the same object. 

Co-authored-by: Marwan Sarieddine <sarieddine.marwan@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
Signed-off-by: Marwan Sarieddine <sarieddine.marwan@gmail.com>
Signed-off-by: Seiji Eicher <seiji@anyscale.com>
 2026-05-01 08:43:39 +01:00
Andreas KaratzasandGitHub 32964e7700 [ROCm][CI] Upgraded UCX and RIXL (#41210) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-05-01 16:40:47 +09:00
Bugen ZhaoGitHubwang.yuqi
 		a07642667d [Bugfix] Pass reasoning parser kwargs to structured output (#41199) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-04-30 23:38:02 -07:00
baonudesifeizhaiandGitHub c3868bbbe4 [compile] Add FlashInfer FP8 async TP fusion and preserve allreduce fusion ordering #27893 (#39505) 
Signed-off-by: baonudesifeizhai <baonudesifeizhai@gmail.com>
Signed-off-by: baonudesifeizhai <85092850+baonudesifeizhai@users.noreply.github.com>
Signed-off-by: roG0d <baonudesifeizhai@gmail.com>
 2026-05-01 05:08:34 +00:00
sychen52andGitHub 947138b6c2 Add nvfp4 kv cache support (#40177) 
Signed-off-by: Shiyang Chen <shiychen@nvidia.com>
 2026-05-01 04:55:16 +00:00
Or OzeriandGitHub 941fb50835 [kv_offload+HMA][12/N]: Scheduler-side support for sliding window groups (#41228) 
Signed-off-by: Or Ozeri <oro@il.ibm.com>
 2026-05-01 06:59:17 +03:00
Juhi MittalGitHubmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		6b6ac6c3c7 [Kernel][MoE] Support GELU on TRT-LLM NvFP4 fused MoE for Gemma4 (#41050) 
Signed-off-by: Juhi Mittal <juhim@nvidia.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-05-01 03:37:43 +00:00
Stefano CastagnettaandGitHub b542bdf7fb [Bugfix] Disable FlashInfer CUTLASS MoE on SM110 (Jetson Thor AGX) (#40808) 
Signed-off-by: Stefano Castagnetta <scastagnetta@nvidia.com>
 2026-04-30 20:08:49 -07:00
Ronen SchafferandGitHub 415a879899 [KV Offload] Use Collection instead of Sequence/Iterable for OffloadingManager key parameters (#41361) 
Signed-off-by: Ronen Schaffer <ronen.schaffer@ibm.com>
 2026-05-01 05:18:38 +03:00
Dong WandGitHub 7198940b39 [Model] Add Moondream3 model support(only query and caption skills) (#32325) 
Signed-off-by: Dong Wang <dongw2019@gmail.com>
 2026-05-01 10:06:48 +08:00
Luis 🚀GitHubAndrew Sansom
 		14043dfecd feat: Enable prompt_embeds Content Part Support in vLLM Chat Completions API (#40720) 
Signed-off-by: Luis Robaina <luis@protopia.ai>
Signed-off-by: Luis Robaina 🚀 <luisfabian1545@gmail.com>
Signed-off-by: LuisRobaina <luis@protopia.ai>
Co-authored-by: Andrew Sansom <qthequartermasterman@gmail.com>
 2026-05-01 10:05:55 +08:00
Andreas KaratzasandGitHub 1adaa5056b [ROCm][CI] Add ROCm score absolute tolerance floor (#41341) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-04-30 18:59:35 -07:00
SoyaazzGitHubMatthew Bonanni
 		4d5c89295b (bugfix): block_size check for flex attn (#41363) 
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-04-30 18:59:26 -07:00
Nick HillandGitHub dd5506a157 [Core] Simplify handling of scheduler_reserve_full_isl option (#41064) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-04-30 18:10:00 -07:00
Yongye ZhuGitHubRoger Wang
 		a3c83ff2fd Faster per-token fp8 group quant packed kernel for blackwell (#41326) 
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
 2026-04-30 18:09:55 -07:00
Woosuk KwonandGitHub 9c61864bf8 [DeepSeek] Use torch.mm for bf16xbf16->fp32 gemm (#41300) 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
 2026-04-30 16:28:57 -07:00
Tran LeandGitHub 71725f6730 [Bugfix] Fix RoutedExpertsCapturer for Gemma 4 MoE (top_k_experts) (#41401) 
Signed-off-by: Tran Le <tranle@fireworks.ai>
 2026-04-30 16:19:59 -07:00
Yongye ZhuGitHubZijing LiuClaude Opus 4.7
 		b4806c8ee1 [DSV4] Add BF16 and MXFP8 A2A support for flashinfer a2a one sided (#40960) 
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Signed-off-by: Zijing Liu <liuzijing2014@gmail.com>
Co-authored-by: Zijing Liu <liuzijing2014@users.noreply.github.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-30 15:33:12 -07:00
Wentao YeandGitHub 526927be94 [Model Runner v2] Fix v2 compile counter num_gpu_runner_capture_triggers and num_cudagraph_captured (#41285) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-04-30 15:20:11 -07:00
Michael GoinandGitHub 75a4c166f2 Fix typo in log message for indexer cache (#41419) 
Signed-off-by: Michael Goin <mgoin64@gmail.com>
 2026-04-30 15:02:14 -07:00
fxmarty-amdGitHubClaude
 		2917d6363a [NVFP4][Hopper/AMD Instinct] Add Triton kernels for NVFP4 dequantization and QDQ emulation (#40033) 
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-04-30 17:35:48 -04:00
Stefano CastagnettaandGitHub efb4cdf2b8 [CI/Build] Skip Prithvi/Terratorch model-registry tests when terratorch is missing (#41389) 
Signed-off-by: Stefano Castagnetta <scastagnetta@nvidia.com>
 2026-04-30 12:47:55 -07:00
Stefano CastagnettaGitHubmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		92a7c121b6 [CI] Add MTP coverage: Qwen3.5 correctness + no-sync spec decode (#40472) 
Signed-off-by: Stefano Castagnetta <scastagnetta@nvidia.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-04-30 12:24:09 -07:00
Jee Jee LiandGitHub 307b17ce33 [DSV4] Avoid redundant dtype conversion. (#41374) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2026-04-30 09:57:27 -07:00
wenjun liuGitHubKunshang Ji
 		3ca6ca210f xpu docker: pin oneAPI to 2025.3 and avoid unintended 2026 upgrade (#41380) 
Signed-off-by: wendyliu235 <wenjun.liu@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-04-30 16:02:23 +00:00
Stefano CastagnettaandGitHub 10558f5f46 [CI/Build] Skip terratorch + torchgeo while PyPI has lightning quarantined (#41377) 
Signed-off-by: Stefano Castagnetta <scastagnetta@nvidia.com>
 2026-04-30 07:59:07 -07:00
tejGitHubTJianHAIAI
 		121dbe7a22 [ROCm] ROCm DeepEP API updated to latest (#39721) 
Signed-off-by: Tej Kiran <vpolamre@amd.com>
Signed-off-by: tej <37236721+itej89@users.noreply.github.com>
Co-authored-by: TJian <tunjian.tan@embeddedllm.com>
Co-authored-by: HAIAI <39548240+HAIAI@users.noreply.github.com>
 2026-04-30 07:46:59 -07:00
Matthew BonanniandGitHub f03d82efdd [UX][Bugfix] Fix OOM by setting PyTorch max_split_size_mb during model loading (#41268) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-04-30 07:46:54 -07:00
Ilya MarkovGitHubMarkov IlyaNicolò Lucchesi
 		a7fb008510 [EPLB] Optimize memory overhead in Nixl communicator (#40013) 
Signed-off-by: ilmarkov <markovilya197@gmail.com>
Signed-off-by: Markov Ilya <markovilya19@gmail.com>
Co-authored-by: Markov Ilya <markovilya19@gmail.com>
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com>
 2026-04-30 07:46:49 -07:00
Harry MellorandGitHub ff449b6426 Stop mergify labelling from skipping pre-commit (#41362) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-04-30 05:48:38 -07:00
Stefano CastagnettaGitHubHarry Mellor
 		3527229517 [Doc] Fix RTD build: pytorch.org/docs/stable/objects.inv returns 404 (#41353) 
Signed-off-by: Stefano Castagnetta <scastagnetta@nvidia.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-04-30 05:06:44 -07:00
Xiaoshuang WangGitHubCyrus Leungmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		b55b26520c [MoE] Make MoERunnerInterface a PluggableLayer for OOT support (#35178) 
Signed-off-by: wxsIcey <1790571317@qq.com>
Signed-off-by: Icey <1790571317@qq.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-04-30 03:31:08 -07:00
snadampalandGitHub 3179e53135 [P/D] Prefill compute optimizations with bi-directional KV cache transfers between P and D nodes (#32553) 
Signed-off-by: Sunita Nadampalli <nadampal@amazon.com>
 2026-04-30 10:14:20 +00:00
Nicolò LucchesiandGitHub efdc95674d [KVConnector] MultiConnector SupportsHMA (#39571) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2026-04-30 02:10:50 -07:00
Chenxi QianGitHubmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		54146a9bf9 [Bugfix] correct h matrix layout in chunk_kda output kernel (#40956) 
Signed-off-by: ChenxiQian <chenxi.qian.cq@outlook.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-04-30 16:22:41 +08:00
BaekpicaGitHubsunghoon.baekOpenAI Codex
 		ca97f7b9bb Fix Gemma4 MoE expert weight remapping (#41206) 
Signed-off-by: sunghoon.baek <sunghoon.baek@connectfy.cloud>
Co-authored-by: sunghoon.baek <sunghoon.baek@connectfy.cloud>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-04-30 00:12:42 -07:00
Ekagra RanjanandGitHub a04e0cf3b8 Fix Cohere ASR after HF upgrade (#40582) 
Signed-off-by: Ekagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com>
 2026-04-29 23:39:04 -07:00
Dhruv SingalGitHubClaude Opus 4.6Dhruv SingalYour NameOpenCodeSimon Mo
 		cb1b02d0e8 [Frontend] Add VLLM_SKIP_MODEL_NAME_VALIDATION environment variable (#34676) 
Signed-off-by: Dhruv Singal <dhruvsingalabc@gmail.com>
Signed-off-by: Dhruv Singal <dsingal@Dhruvs-MacBook-Pro.local>
Signed-off-by: Your Name <you@example.com>
Signed-off-by: vLLM Assistant <assistant@vllm.ai>
Signed-off-by: Simon Mo <simon.mo@hey.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Dhruv Singal <dsingal@Dhruvs-MacBook-Pro.local>
Co-authored-by: Your Name <you@example.com>
Co-authored-by: OpenCode <noreply@openai.com>
Co-authored-by: Simon Mo <simon.mo@hey.com>
 2026-04-29 23:19:09 -07:00
Yongye ZhuGitHubClaude Opus 4.7
 		a749a33d8d [Bugfix] Fix persistent_topk cooperative deadlock at TopK=1024 (#41189) 
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-29 21:03:45 -07:00
Martin HickeyGitHubOr Ozeri
 		c42981d034 [Refactor][kv_offload] KV Offloading maintainability improvements (#40538) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
 2026-04-30 05:55:31 +03:00
Wei ZhaoandGitHub 0ff1bf9bb1 [Bugfix] Fix failure to allocate KV blocks error (#41282) 
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
 2026-04-29 18:44:07 -07:00
Kevin H. LuuGitHubClaude
 		0ab67c0222 [CI] Add key field to all test_areas pipeline steps (#41201) 
Signed-off-by: khluu <khluu000@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-04-29 16:59:16 -07:00
Rohan PotdarandGitHub 3795d7acf4 [ROCm][Bugfix][GPTOSS]: fix input_ids and expert_map args for quark w4a8 gptoss (#41165) 
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com>
 2026-04-29 16:39:01 -07:00
Nick HillandGitHub 18599bfdf2 [Ci][BugFix] Fix slow DP tests due to bad teardown logic (#41166) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-04-29 19:31:00 -04:00
Thien TranandGitHub 296741d025 [DSv4] Use cvt PTX for FP32->FP4 conversion (#41015) 
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
 2026-04-29 16:16:40 -07:00
UranusGitHubClaude Opus 4.6Woosuk Kwon
 		a966aaed30 [Bugfix][MLA] Size arange_buffer to max_num_batched_tokens to prevent CUDA IMA (#39277) 
Signed-off-by: UranusSeven <109661872+UranusSeven@users.noreply.github.com>
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>
 2026-04-29 16:14:50 -07:00
Hemanth AcharyaandGitHub 6841f5dc77 [ROCm] Add env flags to disable dynamic MXFP4 quant and enable AITER tuned GEMMs for Attention Projection Layers (#39987) 
Signed-off-by: Hemanth Acharya <heachary@amd.com>
 2026-04-29 16:07:46 -07:00