Bugen Zhao and khluu
05adde9955
[Bugfix] Pass reasoning parser kwargs to structured output ( #41199 )
...
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: wang.yuqi <yuqi.wang@daocloud.io >
(cherry picked from commit a07642667d
2026-05-01 12:14:44 -07:00
Nick Hill and khluu
a076bd1e67
[Core] Account for num_gpu_blocks_override in max_model_len checks ( #41069 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
(cherry picked from commit e68fa1b90a
2026-04-30 21:39:13 -07:00
Dao007forever and khluu
84c276d7ea
[Bugfix] Cap SWA/chunked-local runtime admission to startup pool-sizing bound ( #40946 )
...
Signed-off-by: Dao Le <Dao007forever@gmail.com >
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
(cherry picked from commit 7b1bc0a3eb
2026-04-26 21:43:00 -07:00
+8
4d51588e23
[Feat] DeepSeek V4 Rebased ( #40860 )
...
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
Signed-off-by: qizixi <zixi@inferact.ai >
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com >
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Yongye Zhu <yongye@inferact.ai >
Co-authored-by: Simon Mo <simon@inferact.ai >
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Giancarlo Delfin <gdelfin@inferact.ai >
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Roger Wang <hey@rogerw.io >
Co-authored-by: Roy Wang <yasong.wang@inferact.ai >
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai >
Co-authored-by: youkaichao <youkaichao@gmail.com >
Co-authored-by: Zhewen Li <jerven.vllm@gmail.com >
Co-authored-by: Zijing Liu <liuzijing2014@gmail.com >
Co-authored-by: khluu <khluu000@gmail.com >
Co-authored-by: qizixi <zixi@inferact.ai >
Co-authored-by: Zhewen Li <zhewenli@inferact.ai >
2026-04-26 18:31:08 -07:00
b39c266dae
[KV Offload] Offload all KV blocks when doing prefill in P/D ( #40346 )
...
Signed-off-by: omerpaz95 <omerpaz95@gmail.com >
Signed-off-by: omerpaz95 <73347585+omerpaz95@users.noreply.github.com >
Co-authored-by: Or Ozeri <or@ozery.com >
2026-04-26 15:06:01 +03:00
Andreas Karatzas and GitHub
95995bbef8
[ROCm][Engine] Fix GPU memory leaks in engine shutdown and test workaround for async KV prefix cache reset ( #38503 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
2026-04-25 05:25:20 +00:00
Andreas Karatzas and GitHub
428b988c98
[ROCm][CI] Fix trust_remote_code AttributeError in EAGLE3 acceptance length test ( #40306 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
2026-04-25 02:59:31 +00:00
626daa2076
[Feat] Unified Synthetic Acceptance Rate for V1 and V2 ( #40662 )
...
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com >
Signed-off-by: Benjamin Chislett <chislett.ben@gmail.com >
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2026-04-24 00:48:08 +00:00
Nick Hill and GitHub
fe85a92e86
[Core] Avoid seq_lens_cpu GPU->CPU sync ( #40654 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
2026-04-24 00:35:55 +00:00
Matthew Bonanni and GitHub
cde8d24710
[Spec Decode] Move SpecDecodeBaseProposer out of eagle.py ( #40732 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
2026-04-23 22:28:27 +00:00
Nicolò Lucchesi and GitHub
8824f50f1f
[CI] Split disaggregated tests into own test-area ( #40623 )
...
Signed-off-by: NickLucche <nlucches@redhat.com >
2026-04-23 23:20:12 +08:00
29f64c5f5e
FlexAttention non-causal support ( #40394 )
...
Signed-off-by: Fynn Schmitt-Ulms <fschmitt@redhat.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-04-22 13:22:57 -07:00
Nicolò Lucchesi and GitHub
33ef1941e2
[Bugfix][CI] Fix v1/kv_connector/unit/test_nixl_connector_hma.py::test_fewer_blocks_with_hma ( #40597 )
...
Signed-off-by: NickLucche <nlucches@redhat.com >
2026-04-22 14:21:02 +01:00
aad88f8486
[kv_offload+HMA][8/N]: Support multi-group worker transfer ( #38453 )
...
Signed-off-by: Or Ozeri <oro@il.ibm.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-04-22 08:44:00 +03:00
96a85c5750
[Startup][UX] Enable CUDAGraph memory profiling by default ( #38284 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com >
2026-04-21 18:16:59 -04:00
Wentao Ye and GitHub
16688b26a6
[Perf] Optimize batch invariant with fused rms norm, 2.1% E2E latency improvement ( #40413 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-04-21 19:51:03 +00:00
zhanqiuhu and GitHub
cc3993b05d
nixl refactor [2/N]: unify TpKVTopology + HeteroTPTransferConfig into TransferTopology ( #39529 )
...
Signed-off-by: Zhanqiu Hu <zhu@redhat.com >
2026-04-20 12:39:08 +02:00
f774ba028a
[kv_offload+HMA][4/N]: Support sliding window lookup ( #36645 )
...
Signed-off-by: Or Ozeri <oro@il.ibm.com >
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com >
2026-04-20 12:53:51 +03:00
omerpaz95 and GitHub
4353c9cb4a
[KV Offload] Pass request context ( #39185 )
...
Signed-off-by: omerpaz95 <omerpaz95@gmail.com >
2026-04-19 08:54:59 +03:00
Yusuf Mohammad and GitHub
b5f6c5f834
Added general ND x ND matmul and unit test for it ( #39909 )
...
Signed-off-by: Yusuf <yusufmohammad@live.com >
2026-04-18 10:05:21 -04:00
Nikita Shapovalov and GitHub
692db29cd4
[Bugfix] Fix Ray compiled-DAG SHM channel stalls by detaching zero-copy np.ndarray logprobs buffers ( #35736 )
...
Signed-off-by: Nikita Shapovalov <nikita@poolside.ai >
2026-04-16 23:49:29 +08:00
Nicolò Lucchesi and GitHub
3daca38e22
[Misc] toy_proxy_server handle min_tokens ( #39706 )
...
Signed-off-by: NickLucche <nlucches@redhat.com >
2026-04-16 15:08:22 +00:00
03f8d3a548
Update to transformers v5 ( #30566 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Signed-off-by: khluu <khluu000@gmail.com >
Signed-off-by: Kevin H. Luu <khluu000@gmail.com >
Signed-off-by: Cyrus Leung <cyrus.tl.leung@gmail.com >
Co-authored-by: khluu <khluu000@gmail.com >
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com >
Co-authored-by: jiang1.li <jiang1.li@intel.com >
2026-04-15 16:29:15 -07:00
Wentao Ye and GitHub
7c636432c6
[CI Bug] fix flaky test ( #39938 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-04-15 17:20:06 -04:00
Wentao Ye and GitHub
39ac640490
[Bug] Fix batch invariant test issue, bs=1 with max_seq_num = 1 ( #39320 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-04-15 16:28:43 -04:00
zhanqiuhu and GitHub
0b790a2501
[Speculative Decoding] Add DFlash speculators config parsing ( #38300 )
...
Signed-off-by: Zhanqiu Hu <zhu@redhat.com >
2026-04-15 16:22:15 -04:00
zhanqiuhu and GitHub
41488f2acd
[Bugfix][NIXL] Fix _logical_to_kernel_block_ids conversion for non-mamba models ( #39724 )
...
Signed-off-by: Zhanqiu Hu <zhu@redhat.com >
2026-04-15 20:08:58 +00:00
55e1a8e103
[Mooncake] Fix mixed MLA+Eagle block-size validation ( #39596 )
...
Signed-off-by: Zhewen Li <zhewenli@inferact.ai >
Co-authored-by: Zhewen Li <zhewenli@inferact.ai >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-04-15 11:36:47 -07:00
Csrayz and GitHub
68be0f853e
[Metrics] Add request_id to FinishedRequestStats to enable correlation between metrics and requests ( #39710 )
...
Enables external `StatLogger` plugins to correlate per-request metrics
with request-level context. Also, this is a pre-requisite for Prometheus
exemplars in #30972 .
Signed-off-by: Csrayz <33659823+Csrayz@users.noreply.github.com >
2026-04-15 11:24:17 +00:00
Or Ozeri and GitHub
235e1f930a
[kv_offload+HMA][3/N]: Remove block_size from KVEvents ( #36644 )
...
Signed-off-by: Or Ozeri <oro@il.ibm.com >
2026-04-15 11:53:19 +03:00
wliao2 and GitHub
3abf858443
[Test] Refactor hard coded device string in test files under compile/quantization/models/model_executor folders ( #38901 )
...
Signed-off-by: Liao, Wei <wei.liao@intel.com >
2026-04-15 11:02:35 +08:00
zhanqiuhu and GitHub
30679319e8
[CI][KVConnector][Metrics] Update multi KV connector edge case according to prefill stats changes ( #39808 )
...
Signed-off-by: Zhanqiu Hu <zhu@redhat.com >
2026-04-14 18:59:15 +00:00
be0c855ebd
[KV Offload] Unified memory layout for offloading workers ( #37206 )
...
Signed-off-by: omerpaz95 <omerpaz95@gmail.com >
Co-authored-by: Or Ozeri <oro@il.ibm.com >
2026-04-14 21:33:33 +03:00
80118853f4
[MM][Perf][CG] Support ViT full CUDA graph for Qwen3-VL video inference ( #38061 )
...
Signed-off-by: shen-shanshan <467638484@qq.com >
Signed-off-by: Shanshan Shen <87969357+shen-shanshan@users.noreply.github.com >
Co-authored-by: Roger Wang <hey@rogerw.io >
2026-04-14 16:49:32 +08:00
d3af8c1831
[Core][Metrics][BugFix] Replace num_cached_tokens/num_external_computed_tokens with PrefillStats ( #37460 )
...
Related to `Counters can only be incremented by non-negative amounts`
error with the `vllm:prompt_tokens_by_source_total` metric.
Signed-off-by: Mark McLoughlin <markmc@redhat.com >
Co-authored-by: Or Ozeri <or@ozery.com >
2026-04-14 09:00:45 +01:00
8213e8f880
Bug/test eagle dp v0 ( #38938 )
...
Signed-off-by: Monishver Chandrasekaran <monishverchandrasekaran@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com >
2026-04-13 20:50:08 +00:00
5c18b961d6
[Core][Metrics] expose waiting request breakdown via labeled metric (capacity/deferred) ( #38435 )
...
Signed-off-by: Mukesh Baphna <mukesh@hippocraticai.com >
Signed-off-by: Mark McLoughlin <markmc@redhat.com >
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com >
Co-authored-by: Mark McLoughlin <markmc@redhat.com >
2026-04-13 15:30:55 -04:00
zhanqiuhu and GitHub
10d9872d3a
[CI][Metrics] Fix local_cache_hit assertion after prompt tokens metrics updates ( #39709 )
...
Signed-off-by: ZhanqiuHu <zhu@redhat.com >
2026-04-13 15:16:56 +00:00
Nicolò Lucchesi and GitHub
3244a2ebf2
[KVConnector][NIXL] Organize NIXL connector into its own directory ( #39354 )
...
The number of features supported by the connector has grown substantially
and the `nixl_connector.py` file has accumulated a lot of code. Creates a separate
directory and isolates connector/scheduler code in the hope of improving clarity
and maintainability.
Further refactor of components aimed at improving clarity and simplifying code
will follow soon.
Signed-off-by: NickLucche <nlucches@redhat.com >
2026-04-12 13:10:50 +00:00
Mark McLoughlin and GitHub
72ff142c37
[Core][Metrics] Remove vllm:prompt_tokens_recomputed metric ( #38709 )
...
Signed-off-by: Mark McLoughlin <markmc@redhat.com >
2026-04-12 12:22:01 +03:00
cc07dad789
[HMA] [KVEvent] Enable GPU-side KV events for HMA ( #37688 )
...
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com >
Co-authored-by: Or Ozeri <or@ozery.com >
2026-04-12 10:01:02 +03:00
0f3ce4c74b
[XPU] Fix spec-decode UTs under tests/v1/spec_decode ( #38491 )
...
Signed-off-by: Yan Ma <yan.ma@intel.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-04-11 01:31:00 +00:00
zhanqiuhu and GitHub
e281cb721c
[CI] Add MultiConnector (Nixl+Offloading) e2e edge case tests ( #39343 )
...
Signed-off-by: ZhanqiuHu <zhu@redhat.com >
2026-04-10 17:35:03 +00:00
Nicolò Lucchesi and GitHub
c9dddc144b
[CI] Add Nixl+OffloadingConnector e2e integration tests ( #39200 )
...
Signed-off-by: NickLucche <nlucches@redhat.com >
2026-04-10 21:40:40 +08:00
Yan Ma and GitHub
ec68d53b2b
Add platform manual_seed_all API ( #38468 )
...
Signed-off-by: Yan Ma <yan.ma@intel.com >
2026-04-10 13:43:50 +08:00
Chauncey and GitHub
ecbfbb8d61
[Feature] Add auto-detection for reasoning_config when only reasoning_parser is set ( #38214 )
...
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com >
2026-04-10 01:36:26 +00:00
a8c6ee9b78
[Performance Improvement] Update batched_count_greater_than to handle batch size 1 without recompile ( #38933 )
...
Signed-off-by: Lucas Kabela <lucaskabela@meta.com >
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com >
2026-04-09 23:51:31 +08:00
wang.yuqi and GitHub
66c079ae83
[Frontend][4/n] Improve pooling entrypoints | pooling. ( #39153 )
...
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
2026-04-09 10:09:45 +00:00
e80e633927
[XPU] Skip VLLM_BATCH_INVARIANT for XPU in EAGLE DP test ( #39164 )
...
Signed-off-by: sihao.li <sihao.li@intel.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-04-09 12:45:16 +08:00
Chendi.Xue and GitHub
ef5a226819
[PD][HeteroArch]Fix accuracy issue with CPU_ATTN as Decoder and Flash_ATTN as prefiller ( #38935 )
...
Signed-off-by: Chendi Xue <chendi.xue@intel.com >
2026-04-09 11:19:07 +08:00
