 MattandGitHub
|
a19ff2218a
|
[Hardware][AMD][CI] Fix Spec Decode Eagle test group (#46018)
Signed-off-by: Matthew Wong <Matthew.Wong2@amd.com>
|
2026-06-21 17:40:02 -05:00 |
|
|
Benjamin ChislettandGitHub
|
89bd2c14d3
|
[Spec Decode] Add Qwen3 architecture support for EAGLE3 (#43132)
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
|
2026-06-21 13:55:26 -07:00 |
|
|
|
c441ad1c07
|
[KV Offloading] Add labeled metrics support (#45957)
Signed-off-by: srinivas_oo7 <sklinkedin0120@gmail.com>
Co-authored-by: srinivas_oo7 <sklinkedin0120@gmail.com>
|
2026-06-21 18:04:01 +00:00 |
|
|
|
2cac89f9da
|
[Spec Decode] Support mixed KV page sizes for DFlash (#45181)
Signed-off-by: Alex Steiner <asteiner@nvidia.com>
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Giancarlo Delfin <gdelfin@inferact.ai>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
|
2026-06-21 22:45:14 +08:00 |
|
|
|
b91b7726e0
|
[ROCm][P/D] Support MiniMax-M3 mixed KV layouts in MoRIIO READ mode (#46039)
Signed-off-by: Jun Kang Chow <junkangchow@gmail.com>
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: Hongxia Yang <hongxia.yang@amd.com>
Co-authored-by: Tan Pin Siang <tanpinsiang@gmail.com>
Co-authored-by: vllmellm <vllm.ellm@embeddedllm.com>
Co-authored-by: Chun Fang <chun.fang@amd.com>
Co-authored-by: TianDi101 <ditian12@amd.com>
Co-authored-by: functionstackx <47992694+functionstackx@users.noreply.github.com>
Co-authored-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-06-21 12:55:19 +00:00 |
|
|
Palaiologos1453andGitHub
|
d3ad8e8bcd
|
[Bugfix] Defer offload reads while transfers are pending (#46231)
Signed-off-by: test test <2260891073@qq.com>
|
2026-06-21 14:30:13 +03:00 |
|
|
Nick HillandGitHub
|
7df3d7dada
|
[Core] Ensure memory is pinned prior to async h2d copy (#45424)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
|
2026-06-20 20:02:24 -07:00 |
|
|
|
6e919960af
|
[Perf] Skip/shrink all_token_ids copy in scheduler for non-async and V2 runner (#45840)
Signed-off-by: amanchugh89 <amanchugh.89@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-06-20 22:36:57 +00:00 |
|
|
Jonathan ChenandGitHub
|
c88d3d4775
|
[SimpleCPUOffloadConnector] PCP + DCP support (#39831)
Signed-off-by: Jonathan Chen <chenleejonathan@gmail.com>
|
2026-06-20 15:01:06 -07:00 |
|
|
Yifan QiaoandGitHub
|
ab7fcbdd5d
|
[Perf][KVConnector][Mooncake] Compact chunk-hash keys and zero-copy lookup wire format (#45969)
|
2026-06-20 15:00:11 -07:00 |
|
|
|
3b4a76b63f
|
[KV-Offloading] : Expose CPU cache usage metric (#45737)
Signed-off-by: Varun Sundar Rabindranath <varun-sundar-rabindranath@h100-01.nemg-001.lab.rdu2.dc.redhat.com>
Signed-off-by: <>
Co-authored-by: Varun Sundar Rabindranath <varun-sundar-rabindranath@h100-01.nemg-001.lab.rdu2.dc.redhat.com>
|
2026-06-20 21:21:55 +00:00 |
|
|
|
cc22621b51
|
[KV Offload] Support packed HMA KV cache layout (#46205)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
|
2026-06-20 21:19:40 +00:00 |
|
|
|
dced290769
|
[Hardware][AMD][CI] Fix e2e core test group (#46024)
Signed-off-by: Matthew Wong <Matthew.Wong2@amd.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-06-20 02:04:35 -05:00 |
|
|
|
01192139bf
|
[DSv4] Pack KV caches into contiguous per-block allocations for DeepSeek V4 (#44577)
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-06-19 12:55:42 -04:00 |
|
|
|
41dcf49ca5
|
[Bugfix][KV Connector] Disable Mooncake TP put-striding when DCP > 1 (#45371)
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Jingyi Yang <girasoleyang@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-06-18 15:13:44 -07:00 |
|
|
|
35e4dd4a69
|
[KV Connector][Mooncake] Async lookup to reduce scheduler overhead (#45659)
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-06-18 21:44:02 +00:00 |
|
|
Woosuk KwonandGitHub
|
16908e132e
|
[MRV2] Make FP32 Gumbel sampling more accurate (#45996)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
|
2026-06-18 19:42:09 +00:00 |
|
|
|
ea6078fe6a
|
[KV Connector][Offloading] Disable parallel-agnostic fs-tier cache on V2 model runner (#46044)
Signed-off-by: Itay Etelis <etelis2019@gmail.com>
Co-authored-by: Itay Etelis <etelis2019@gmail.com>
|
2026-06-18 20:43:35 +03:00 |
|
|
stefankoncarevicandGitHub
|
e2352c2974
|
[ROCm][Spec Decode] Fix probabilistic draft probs test attention backend (#45706)
Signed-off-by: Stefan Koncarevic <stefan.koncarevic@amd.com>
|
2026-06-18 11:59:37 -05:00 |
|
|
AsharandGitHub
|
4cb5e746b6
|
[Rust Frontend]: Add /get_world_size route with static parallel size (#44801)
|
2026-06-18 13:10:20 +00:00 |
|
|
Wei ZhaoandGitHub
|
5fd3b276f8
|
[Mooncake] Skip KV lookup for non-reachable SWA blocks (#45444)
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
|
2026-06-18 02:23:20 -07:00 |
|
|
|
554352a311
|
[Test][KV Connector] Add request_finished fence population tests for offloading scheduler (#45679)
Signed-off-by: Alex <alex.tech.lab@outlook.com>
Signed-off-by: AlexHuang <jihuihuang@future.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
|
2026-06-18 08:13:52 +03:00 |
|
|
|
421c1ec448
|
[KV Offloading] Remove dummy worker-side stats from OffloadingConnector (#45905)
Signed-off-by: Alex <alex.tech.lab@outlook.com>
Signed-off-by: AlexHuang <jihuihuang@alexai.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
|
2026-06-18 08:13:28 +03:00 |
|
|
Ronen SchafferandGitHub
|
f428718ffe
|
[Fix][KV offload] Defer on_request_finished until in-flight transfers drain (#45823)
Signed-off-by: Ronen Schaffer <ronen.schaffer@ibm.com>
|
2026-06-18 07:05:46 +03:00 |
|
|
|
d57888efa4
|
[SimpleCPUOffloadConnector]: Add support for reset_cache() (#39726)
Signed-off-by: Jonathan Chen <chenleejonathan@gmail.com>
Signed-off-by: Jonathan <chenleejonathan@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-06-17 19:47:12 -07:00 |
|
|
|
ed938ad7db
|
[CPUOffloading] Guard CPU eviction check (#45757)
Signed-off-by: Varun Sundar Rabindranath <varun-sundar-rabindranath@h100-01.nemg-001.lab.rdu2.dc.redhat.com>
Co-authored-by: Varun Sundar Rabindranath <varun-sundar-rabindranath@h100-01.nemg-001.lab.rdu2.dc.redhat.com>
|
2026-06-18 05:34:59 +03:00 |
|
|
|
0d339cf135
|
[Bugfix] Fix NixlConnector handshake block_len validation for GQA-replicated KV heads (#45879)
Signed-off-by: Oseltamivir <58582368+Oseltamivir@users.noreply.github.com>
Co-authored-by: waynehacking8 <waynehacking8@gmail.com>
|
2026-06-17 15:11:29 -07:00 |
|
|
|
6d8fff5698
|
[KV Connector][Offloading] Avoid blocking the engine to flush offloads on idle (#45595)
Signed-off-by: Itay Etelis <itay.etelis@ibm.com>
Signed-off-by: Or Ozeri <oro@il.ibm.com>
Signed-off-by: Itay Etelis <Itay.etelis@gmail.com>
Co-authored-by: Itay Etelis <itay.etelis@ibm.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
Co-authored-by: Itay Etelis <Itay.etelis@gmail.com>
|
2026-06-17 11:35:07 +03:00 |
|
|
|
7b5d60cc37
|
[Bugfix][V1] Clean up compiled-model bytecode hooks on VllmRunner exit (#45195)
Signed-off-by: Ting Sun <suntcrick@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-06-16 20:31:17 -07:00 |
|
|
|
556b063e45
|
[XPU] Fix test_spec_decode_logprobs: use FLASH_ATTN for XPU in GPU_DETERMINISM_KWARGS (#44468)
Signed-off-by: Chaojun Zhang <chaojun.zhang@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
|
2026-06-17 11:07:04 +08:00 |
|
|
|
4bf699d310
|
[Kernel] Support DS Mamba tail copy for MTP align mode (#45473)
Signed-off-by: Sungsoo Ha <sungsooh@nvidia.com>
Co-authored-by: Thomas Parnell <tom.parnell@gmail.com>
|
2026-06-16 22:50:30 +00:00 |
|
|
Stan WozniakandGitHub
|
520828789c
|
Apply LRU policy only to proper cache entries (#42656)
Signed-off-by: Stanislaw Wozniak <stw@zurich.ibm.com>
|
2026-06-16 21:49:15 +00:00 |
|
|
|
9d4dc4ca2f
|
[Kernel] Support GLM-5 dimensions for TRT-LLM ragged MLA prefill (#43525)
Signed-off-by: Mohammad Miadh Angkad <176301910+mmangkad@users.noreply.github.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
|
2026-06-16 20:49:47 +00:00 |
|
|
Nick HillandGitHub
|
d8d95998dc
|
[Core] Add prefill step cadence for better non-PD DP balancing (#44558)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
|
2026-06-16 13:17:18 -07:00 |
|
|
|
8e27a9c215
|
[PERF] Fuse multi-group block table staged writes (#44944)
Signed-off-by: jesse <szxfml@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-06-16 10:53:27 -07:00 |
|
|
|
44b2512767
|
[KV Connector][Mooncake] Add cache_prefix to namespace store keys (#45767)
Signed-off-by: Dao Le <Dao007forever@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
2026-06-16 10:24:20 -07:00 |
|
|
|
188c68798e
|
[KVConnector][MoRIIO] Allow overriding the advertised host IP (#45488)
Signed-off-by: Kourosh Hakhamaneshi <kourosh@anyscale.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
2026-06-16 17:18:37 +00:00 |
|
|
Carl YandGitHub
|
eb04c769d3
|
feat: MLA prefill enable FA4 fp8 output (#43050)
Signed-off-by: Carl You <4531192+carlyou@users.noreply.github.com>
|
2026-06-16 07:10:59 -07:00 |
|
|
Hank HanandGitHub
|
d53f4593ce
|
[KV Connector][Mooncake] Pipeline-parallel support for PD-disaggregated serving with Mooncake connector (#44528)
Signed-off-by: hanhan.hank <hanhan.hank@bytedance.com>
Signed-off-by: Hank Han <hanhan7630@outlook.com>
|
2026-06-16 04:35:38 -07:00 |
|
|
|
ad32608e24
|
[MM][Perf][CG] Support dual-path ViT full CUDA graph for DeepSeek-OCR (#43586)
Signed-off-by: shen-shanshan <467638484@qq.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
|
2026-06-16 04:35:20 -07:00 |
|
|
|
7ad894c86a
|
[Bugfix] Prevent cuMemcpyBatchAsync segfault with MTP and KV offloading (#44784)
Signed-off-by: joshua <joshua.abraham@multicorewareinc.com>
Co-authored-by: joshua <joshua.abraham@multicorewareinc.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
|
2026-06-16 07:58:39 +00:00 |
|
|
|
d467a2a7f2
|
[Bugfix] Defer block freeing until in-flight steps finish under async scheduling + PD KV consumer (#45357)
Signed-off-by: llx-08 <2596671364@qq.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com>
|
2026-06-15 21:36:09 +00:00 |
|
|
|
7e612a0f06
|
[KV Offloading] Implement reset_cache for TieringOffloadingManager (#44541)
Signed-off-by: Ronen Schaffer <ronen.schaffer@ibm.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-06-15 18:42:53 +00:00 |
|
|
SaddssandGitHub
|
588db18362
|
[Bugfix] Two-phase KV allocation for cross-group prefix cache hits (supersedes #33775) (#44409)
Signed-off-by: Saddss <2872669061@qq.com>
|
2026-06-15 22:39:59 +08:00 |
|
|
Yejing LaiandGitHub
|
9872921c5f
|
[XPU] skip UT test_with_ngram_gpu_spec_decoding (#44423)
Signed-off-by: Lai, Yejing <yejing.lai@intel.com>
|
2026-06-15 08:46:30 +00:00 |
|
|
|
4ef4492e9b
|
[V1][Spec Decode] Add Dynamic SD (#32374)
Signed-off-by: Ekagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com>
Signed-off-by: Benjamin Chislett <chislett.ben@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com>
|
2026-06-14 00:14:27 -07:00 |
|
|
|
521b88c29e
|
[Bugfix] Reject structured outputs for diffusion decoders with a clear error (#45468)
Signed-off-by: Wayne Chiu <waynehacking8@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-06-13 12:04:01 -07:00 |
|
|
Juan Pérez de AlgabaandGitHub
|
470229c37e
|
[Security] Fix DoS via prompt_embeds on M-RoPE models (#45252)
Signed-off-by: jperezde <jperezde@redhat.com>
|
2026-06-13 10:17:38 +00:00 |
|
|
|
2b3006076c
|
[Security] Add timeout guard for regex compilation in structured outp… (#45118)
Signed-off-by: jperezde <jperezde@redhat.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
|
2026-06-13 09:52:56 +00:00 |
|
|
Nick HillandGitHub
|
1a369783e9
|
[BugFix] Avoid prematurely freeing cached mm encoder outputs (#45347)
Signed-off-by: Roger Wang <hey@rogerw.io>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
|
2026-06-12 15:39:40 -07:00 |
|