dc8df110bc
add warning when FP8 KV cache misses prefill query quantization ( #39752 )
...
Signed-off-by: Michael Goin <mgoin64@gmail.com >
Co-authored-by: Albert Cheng (Engrg-Hardware 1) <albecheng@login-lyris02.lyris.clusters.nvidia.com >
Co-authored-by: Michael Goin <mgoin64@gmail.com >
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2026-04-14 14:43:05 -04:00
be0c855ebd
[KV Offload] Unified memory layout for offloading workers ( #37206 )
...
Signed-off-by: omerpaz95 <omerpaz95@gmail.com >
Co-authored-by: Or Ozeri <oro@il.ibm.com >
2026-04-14 21:33:33 +03:00
Andrew Barnes and GitHub
e64b39ea71
[ROCm] Align AiterFlashAttentionImpl attn_type check with backend ( #39119 )
...
Signed-off-by: Bortlesboat <bortstheboat@gmail.com >
2026-04-14 10:36:26 -07:00
Alessandro Sangiorgi and GitHub
2faad08362
[compile] Nest inductor cache under AOT compile dir ( #39718 )
...
Signed-off-by: Alessandro Sangiorgi <asangior@redhat.com >
2026-04-14 17:17:54 +00:00
Rohan Potdar and GitHub
23f3760217
[Bugfix][ROCm]: Allow gpt_oss_mxfp4 quantization method on rocm ( #39754 )
...
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com >
2026-04-14 17:10:04 +00:00
Mark McLoughlin and GitHub
906a8c15d0
[Core][Metrics] Remove unused SchedulerStats.encoder_cache_usage ( #39693 )
...
Signed-off-by: Mark McLoughlin <markmc@redhat.com >
2026-04-14 12:53:57 -04:00
Micah Williamson and GitHub
4f4f8eaa78
[ROCm][CI] Fix condition for test_per_token_group_quant_fp8_packed ( #39730 )
...
Signed-off-by: Micah Williamson <micah.williamson@amd.com >
2026-04-14 16:14:31 +00:00
Netanel Haber and GitHub
b6890a120a
Bugfix: use_existing_torch.py: Glob recursive subdirs in requirements ( fixes #39024 ) ( #39793 )
...
Signed-off-by: Netanel Haber <58652339+netanel-haber@users.noreply.github.com >
2026-04-14 23:11:46 +08:00
Lucas Kabela and GitHub
c08f3b2a62
Measure encoder compile time seperate from llm backbone ( #39240 )
...
Signed-off-by: Lucas Kabela <lucaskabela@meta.com >
2026-04-14 10:52:49 -04:00
Hexiang Wang and GitHub
f02b3269e7
[PluggableLayer][3/N] Apply PluggableLayer to moe-related layers. ( #33556 )
...
Signed-off-by: whx-sjtu <2952154980@qq.com >
2026-04-14 09:55:00 -04:00
e1e318af01
[MoE Refactor] Remove MoE DP chunking ( #39107 )
...
Signed-off-by: Bill Nell <bnell@redhat.com >
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
2026-04-14 09:48:05 -04:00
f7e62e3d66
[Bugfix] Fix mismatch between global and local attention heads in tensor-parallel mode for param2moe model ( #39707 )
...
Signed-off-by: bhargav-patel-29 <bhargav.patel@tihiitb.org >
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2026-04-14 20:13:36 +08:00
18b1c77211
fix: handle ImportError in load_audio ( #39473 )
...
Signed-off-by: Yiyang Liu <37043548+ianliuy@users.noreply.github.com >
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com >
2026-04-14 19:09:06 +08:00
Matthias Gehre and GitHub
1e4748c66a
[Bugfix] Fix vllm bench serve to count multimodal tokens in "total input tokens" ( #38654 )
...
Signed-off-by: Matthias Gehre <matthias.gehre@amd.com >
2026-04-14 11:00:40 +00:00
6f786f2c50
[Bugfix][Model] Fix Devstral Small 2 HF format weight loading ( #39293 )
...
Signed-off-by: thomasmaindron <thomasmaindron@users.noreply.github.com >
Co-authored-by: thomasmaindron <thomasmaindron@users.noreply.github.com >
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
2026-04-14 10:11:18 +00:00
fxmarty-amd and GitHub
4eee77b877
[fix][MOE] Fix MOE experts intermediate_size dimension not being narrowed before weight loading ( #39688 )
...
Signed-off-by: Felix Marty <Felix.Marty@amd.com >
2026-04-14 09:35:28 +00:00
xiangdong and GitHub
a1993b96fd
[XPU][CI] Remove Arc in label-xpu ( #39776 )
...
Signed-off-by: zengxian <xiangdong.zeng@intel.com >
2026-04-14 02:27:38 -07:00
Julien Debache and GitHub
893b2affff
feat: add TxtSlicesDataset to allow sampling slices from txt file for benchmarking ( #30156 )
...
Signed-off-by: jdebache <jdebache@nvidia.com >
2026-04-14 09:20:03 +00:00
80118853f4
[MM][Perf][CG] Support ViT full CUDA graph for Qwen3-VL video inference ( #38061 )
...
Signed-off-by: shen-shanshan <467638484@qq.com >
Signed-off-by: Shanshan Shen <87969357+shen-shanshan@users.noreply.github.com >
Co-authored-by: Roger Wang <hey@rogerw.io >
2026-04-14 16:49:32 +08:00
c0ecaed950
[Frontend] Offload blocking preprocessing & postprocessing ops to thread pool for pooling entrypoints. ( #39763 )
...
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
Signed-off-by: wang.yuqi <noooop@126.com >
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2026-04-14 08:29:25 +00:00
0008729abf
[Model] Use mm_features for Ernie-4.5 VL M-RoPE ( #39753 )
...
Signed-off-by: Lalit Laxminarayan Bangad <lalitbangad@gmail.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-04-14 01:11:52 -07:00
d3af8c1831
[Core][Metrics][BugFix] Replace num_cached_tokens/num_external_computed_tokens with PrefillStats ( #37460 )
...
Related to `Counters can only be incremented by non-negative amounts`
error with the `vllm:prompt_tokens_by_source_total` metric.
Signed-off-by: Mark McLoughlin <markmc@redhat.com >
Co-authored-by: Or Ozeri <or@ozery.com >
2026-04-14 09:00:45 +01:00
noobHappylife and GitHub
25b3242d8b
Fix Responses API streaming for multiple auto tool calls ( #39626 )
...
Signed-off-by: noobhappylife <aratar1991@hotmail.com >
2026-04-14 13:28:43 +08:00
b075604da1
[Bugfix] Fix Gemma4 tool parser converting bare null to string "null" ( #39679 )
...
Signed-off-by: KimuGenie <baby11686@naver.com >
Signed-off-by: Chauncey <chaunceyjiang@gmail.com >
Co-authored-by: Chauncey <chaunceyjiang@gmail.com >
2026-04-14 04:44:46 +00:00
Flora Feng and GitHub
db8a6d66bf
[Refactor][Parser] Migrate chat completion auto-tool/reasoning/plain streaming to parse_delta ( #39446 )
...
Signed-off-by: sfeng33 <4florafeng@gmail.com >
2026-04-14 04:39:45 +00:00
Chauncey and GitHub
d2130a47bb
[Bugfix]: Fix MinimaxM2ToolParser missing tools parameter ( #39683 )
...
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com >
2026-04-14 11:16:39 +08:00
c687bf226a
[LMCache][MP] optimize save when mla enabled ( #38810 )
...
Signed-off-by: idellzheng <idellzheng@tencent.com >
Co-authored-by: Yihua Cheng <yihua98@uchicago.edu >
2026-04-13 17:56:43 -07:00
Giancarlo Delfin and GitHub
ccf90ba784
[Model Runner V2] Add full cuda graph support for eagle prefill ( #37588 )
...
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai >
2026-04-13 16:01:24 -07:00
Netanel Haber and GitHub
6adacfcb65
ParakeetExtractor performance and UX enhancements ( #39423 )
...
Signed-off-by: Netanel Haber <58652339+netanel-haber@users.noreply.github.com >
2026-04-13 21:37:35 +00:00
Flora Feng and GitHub
14cb86c187
[Refactor][Parser] Simplify parse_delta ( #39728 )
...
Signed-off-by: sfeng33 <4florafeng@gmail.com >
2026-04-13 21:02:13 +00:00
8213e8f880
Bug/test eagle dp v0 ( #38938 )
...
Signed-off-by: Monishver Chandrasekaran <monishverchandrasekaran@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com >
2026-04-13 20:50:08 +00:00
Pedram Razavi and GitHub
3693f922ff
[Bugfix][Pooling] Fix silent weight corruption with buffer-reusing iterators ( #39650 )
...
Signed-off-by: Pedram Razavi <pedram.razavi@gmail.com >
2026-04-13 19:37:27 +00:00
5c18b961d6
[Core][Metrics] expose waiting request breakdown via labeled metric (capacity/deferred) ( #38435 )
...
Signed-off-by: Mukesh Baphna <mukesh@hippocraticai.com >
Signed-off-by: Mark McLoughlin <markmc@redhat.com >
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com >
Co-authored-by: Mark McLoughlin <markmc@redhat.com >
2026-04-13 15:30:55 -04:00
f72b20976c
[Bugfix] Reject non-nvfp4 dtypes when using the flashinfer_nvlink_one_sided all2all backend ( #39717 )
...
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com >
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
2026-04-13 19:13:51 +00:00
610a3efcaf
[Doc] Fix Python-only build 404 fallback guidance ( #38052 )
...
Signed-off-by: George-ao <yuyiao772@gmail.com >
Signed-off-by: Yuyi Ao <yuyiao772@gmail.com >
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-04-13 12:09:31 -07:00
JartX and GitHub
f414f90601
[Bugfix][Kernel][ROCm] Fix triton_w4a16 scales mismatch when BLOCK_K > group_size ( #39705 )
...
Signed-off-by: JartX <sagformas@epdcenter.es >
2026-04-13 14:29:45 -04:00
Nicolò Lucchesi and GitHub
8625ec267b
[Misc] Multi-turn benchmark output performance json ( #39572 )
...
Signed-off-by: NickLucche <nlucches@redhat.com >
2026-04-13 18:15:23 +00:00
995e9a209e
[Bugfix] Use is_integrated to detect UMA GPUs for memory reporting ( #35356 )
...
Signed-off-by: haosdent <haosdent@gmail.com >
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk >
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk >
2026-04-13 11:07:40 -07:00
Yongye Zhu and GitHub
739e5945dc
[Quantization] [Refactor] Create special "GptOssMxfp4MoeMethod" ( #39604 )
...
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
2026-04-13 12:53:58 -04:00
Santino Ramos and GitHub
4d042ed85f
[Bugfix] Fix tensor shape mismatch in sparse attention with speculative decoding ( #39542 )
...
Signed-off-by: Santino Ramos <santinor@inferact.ai >
2026-04-13 08:57:38 -07:00
zhanqiuhu and GitHub
10d9872d3a
[CI][Metrics] Fix local_cache_hit assertion after prompt tokens metrics updates ( #39709 )
...
Signed-off-by: ZhanqiuHu <zhu@redhat.com >
2026-04-13 15:16:56 +00:00
ccd0d1d906
[Bug] Fix rocm sparse attn indexer issue ( #39225 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
Co-authored-by: TJian <tunjian.tan@embeddedllm.com >
2026-04-13 07:53:45 -07:00
Yi Liu and GitHub
d8ddb31644
[Bugfix][CT] Fix KV cache scale handling ( #39418 )
...
Signed-off-by: yiliu30 <yi4.liu@intel.com >
2026-04-13 10:50:16 -04:00
Ekagra Ranjan and GitHub
1ce0318c68
[Bugfix] stream failure when model name not in audio endpoints ( #36679 )
...
Signed-off-by: Ekagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com >
2026-04-13 14:20:07 +00:00
Tihomir Elek and GitHub
8d825b87d6
[Bug] Fix TypeError when hf_config.architectures is None during model loading ( #38849 )
...
Signed-off-by: Tihomir Elek <tiho.elek@gmail.com >
2026-04-13 12:13:21 +01:00
zofia and GitHub
1b19bd7589
[MXFP8] [XPU] add a new compressed tensor schema and add a xpu mxfp8 gemm kernel ( #38707 )
...
Signed-off-by: Zhu, Zufang <zufang.zhu@intel.com >
2026-04-13 16:59:20 +08:00
200a727e94
[Bugfix] Fix Responses API instructions leaking through previous_response_id ( #37727 )
...
Signed-off-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com >
Co-authored-by: Chauncey <chaunceyjiang@gmail.com >
2026-04-13 08:46:33 +00:00
edbc1abd1c
feat: add max_tokens_per_doc in rerank request. ( #38827 )
...
Signed-off-by: Jesus Federico <jefp@amazon.com >
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
Co-authored-by: wang.yuqi <yuqi.wang@daocloud.io >
2026-04-13 01:24:09 -07:00
Flora Feng and GitHub
0e39202ca9
[Bugfix] Fix GLM tool parser streaming with MTP or stream interval ( #39253 )
...
Signed-off-by: sfeng33 <4florafeng@gmail.com >
2026-04-13 05:10:30 +00:00
9dd5ee0117
[XPU]Enhance environment collection for Intel XPU and optimize layout ( #35698 )
...
Signed-off-by: sihao.li <sihao.li@intel.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-04-13 12:51:46 +08:00
