13bf242100
[Feat][KVConnector] Add bind_gpu_block_pool() to KVConnectorBase_V1 ( #39654 )
...
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-05-13 02:10:29 -07:00
9ce74042d3
[Bugfix][SimpleCPUOffloadBackend] Dedup in-flight CPU offload stores across scheduler steps ( #41289 )
...
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-05-13 01:53:32 -07:00
Shanshan Shen and GitHub
92def124bc
[MM][Perf][CG] Support ViT full CUDA graph for Qwen3.5 ( #42151 )
...
Signed-off-by: shen-shanshan <467638484@qq.com >
2026-05-13 16:00:32 +08:00
Nicolò Lucchesi and GitHub
71bcd02ef3
[Bugfix][PD] Fix multi-node TP (TP>8) ( #39907 )
...
Signed-off-by: NickLucche <nlucches@redhat.com >
2026-05-12 22:20:57 -07:00
bnellnm and GitHub
18f6bf5a21
[MoE Refactor] Add sequence parallel tests to test_moe_layer.py ( #41299 )
...
Signed-off-by: Bill Nell <bnell@redhat.com >
2026-05-12 21:52:19 -04:00
+4
ebeb09d822
[KV Transfer] Add MooncakeStoreConnector for KV cache offloading via Mooncake distributed store ( #40900 )
...
Signed-off-by: leichao.lc <leichao.lc@antgroup.com >
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: leichao.lc <leichao.lc@antgroup.com >
Co-authored-by: ivanium <yifanqiao@inferact.ai >
Co-authored-by: aoshen524 <aoshen@inferact.ai >
Co-authored-by: Dao007forever <daole@inferact.ai >
Co-authored-by: Teng Ma <sima.mt@alibaba-inc.com >
Co-authored-by: Pz1116 <zpbzpb123123@gmail.com >
Co-authored-by: foraxe <1055696449@qq.com >
Co-authored-by: Skywalker-EP <173423846@qq.com >
Co-authored-by: fems14 <1804143737@qq.com >
Co-authored-by: jianzs <zheng.shoujian@outlook.com >
Co-authored-by: baxingpiaochong <771405853@qq.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-05-12 16:09:10 -07:00
Nick Hill and GitHub
fe8b42e80c
[CI] Fix test_async_scheduling.py flakiness ( #42455 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
2026-05-12 21:38:32 +00:00
Giancarlo Delfin and GitHub
fe5b4e0fe7
[Model Runner V2] Apply synthetic mode to probabilistic rejection sampler ( #41035 )
2026-05-12 13:37:03 -07:00
KaivalyaMDabhadkar and GitHub
67c89fe40a
[Model][Bugfix] Fix Step3-VL image_embeds input path ( #42333 )
...
Signed-off-by: Kaivalya Dabhadkar <kdabhadkar@nvidia.com >
2026-05-12 18:47:55 +00:00
d9b4990783
[MoE Refactor] EPLB refactoring for FusedMoE ( #41055 )
...
Signed-off-by: Bill Nell <bnell@redhat.com >
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
2026-05-12 14:16:31 -04:00
4d591db470
[MoE Refactor] Introduce RoutedExperts alias for FusedMoE and don't store SharedExperts in MK ( #40735 )
...
Signed-off-by: Bill Nell <bnell@redhat.com >
Signed-off-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
Signed-off-by: Robert Shaw <robertgshaw2@gmail.com >
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
Co-authored-by: Robert Shaw <robertgshaw2@gmail.com >
2026-05-12 13:37:44 -04:00
yzong-rh and GitHub
6ff7405b81
[Bugfix] [Frontend] Responses API, fix merging of messages ( #42189 )
...
Signed-off-by: Yifan Zong <yzong@redhat.com >
Signed-off-by: Yifan <yzong@redhat.com >
2026-05-12 16:09:59 +00:00
Yan Ru Pei and GitHub
bcb9c133ba
feat(kv-events): emit KV cache metadata ( #40984 )
...
Signed-off-by: PeaBrane <yanrpei@gmail.com >
2026-05-12 15:58:48 +00:00
Martin Hickey and GitHub
418ba8ef14
[kv_offload][BugFix] Fix store deferral ( #41945 )
...
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com >
2026-05-12 18:04:44 +03:00
289cee0473
[vLLM IR] Minor improvements ( #39362 ) ( #39558 )
...
Signed-off-by: Avishek Goswami <avishek.goswami@ibm.com >
Co-authored-by: Avishek Goswami <avishek.goswami@ibm.com >
2026-05-12 10:58:36 -04:00
shanjiaz and GitHub
6ccb10d794
Added peagle speculators support ( #41826 )
...
Signed-off-by: shanjiaz <zsjwpianpian@gmail.com >
2026-05-12 07:55:57 -07:00
7a9cc5e7f0
[Model] Support MiniCPM-V 4.6 ( #41254 )
...
Signed-off-by: caitianchi <caitianchi@tc-mb.com >
Signed-off-by: tc-mb <157115220+tc-mb@users.noreply.github.com >
Co-authored-by: caitianchi <caitianchi@tc-mb.com >
2026-05-12 14:28:10 +00:00
593d5a4033
[Bugfix] Fix mismatched kernel-per-logical blocks in NIXL HMA transfer ( #42097 )
...
Signed-off-by: ZhanqiuHu <zhu@redhat.com >
Signed-off-by: Zhanqiu Hu <zhu@redhat.com >
Signed-off-by: NickLucche <nlucches@redhat.com >
Co-authored-by: NickLucche <nlucches@redhat.com >
2026-05-12 15:53:30 +02:00
bnellnm and GitHub
6427603ae8
[MoE Refactor] Move remaining experts classes to experts directory ( #42334 )
...
Signed-off-by: Bill Nell <bnell@redhat.com >
2026-05-12 09:19:46 -04:00
206eaed08d
[MoE Refactor] Move expert map related code into ExpertMapManager class ( #41046 )
...
Signed-off-by: Bill Nell <bnell@redhat.com >
Signed-off-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
Signed-off-by: Robert Shaw <robertgshaw2@gmail.com >
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
Co-authored-by: Robert Shaw <robertgshaw2@gmail.com >
2026-05-12 09:18:27 -04:00
Dipika Sikka and GitHub
a7b801e26d
[MXFP4] Support for linear layers + compressed-tensors integration ( #41664 )
2026-05-12 07:49:33 -04:00
haosdent and GitHub
fc8bf6eedb
[CI] De-flake Language Models Test (Extended Generation) test_models(False-False-5-32-bigcode/starcoder2-3b) ( #42392 )
...
Signed-off-by: haosdent <haosdent@gmail.com >
2026-05-12 10:46:48 +00:00
liuzhenwei and GitHub
07a40ede19
[UT][XPU] fix test_parallel_sampling due to global random state ( #42388 )
...
Signed-off-by: zhenwei-intel <zhenwei.liu@intel.com >
2026-05-12 18:03:23 +08:00
Kevin H. Luu and GitHub
1ff9d33535
[CI] Migrate remaining B200 jobs to b200-k8s with test fixes ( #42387 )
...
Signed-off-by: khluu <khluu000@gmail.com >
2026-05-12 02:00:37 -07:00
7f65f84428
[Bugfix] Fix empty channel/recipient in harmony for /v1/responses ( #35540 )
...
Signed-off-by: kg6-sleipnir <christopherhazen42@gmail.com >
Signed-off-by: chazen <45186108+kg6-sleipnir@users.noreply.github.com >
Co-authored-by: Chauncey <chaunceyjiang@gmail.com >
2026-05-12 08:45:51 +00:00
d37e25ffbe
[Frontend] Consolidate Speech to Text entrypoints. ( #42370 )
...
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
Signed-off-by: wang.yuqi <noooop@126.com >
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2026-05-12 07:06:57 +00:00
ZiTian Zhao and GitHub
630492da30
[Fix] Gemma4 Mixed-Resolution Image Co-Batching Crash ( #42217 )
...
Signed-off-by: zitian.zhao <zitian.zhao@tencentmusic.com >
2026-05-12 03:13:03 +00:00
d7af6b34d8
[Model Runner V2] Bug fix: logprob dtype int64/int32 issue ( #41761 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-05-11 21:55:43 +00:00
wang.yuqi and GitHub
a0dc7a0f36
[CI] Consolidate Speech to Text tests ( #42274 )
...
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
2026-05-11 19:50:17 +00:00
a721315488
[ROCm][Perf] Fix RMSNorm+Quant fusion for gfx950 (non-fnuz) ( #41825 )
...
Signed-off-by: Frida Andersson <fanderss@amd.com >
Signed-off-by: Chuan Li <chuali@amd.com >
Co-authored-by: Markus Hartikainen <markus.hartikainen@amd.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: Chuan Li <chuali@amd.com >
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com >
Co-authored-by: Frida Andersson <frida-andersson@users.noreply.github.com >
Co-authored-by: TJian <tunjian.tan@embeddedllm.com >
2026-05-11 15:00:51 -04:00
Hexiang Wang and GitHub
7863fff6e5
[ROCm][DSv4] implement flash sparse mla with triton kernels ( #41812 )
...
Signed-off-by: whx-sjtu <xiaowang990929@gmail.com >
2026-05-11 09:27:11 -07:00
Wentao Ye and GitHub
0d453e2336
[Perf] Batch invariance with Cutlass fp8 support, 28.9% E2E latency improvement ( #40408 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com >
2026-05-11 12:20:58 -04:00
4955990f1b
[kv_offload] Move FilterReusedOffloadingManager logic to CPUOffloadingManager ( #41727 )
...
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com >
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2026-05-11 18:09:29 +03:00
724ed2fc35
[DSv4] Improved dequant gather K cache kernel ( #42236 )
...
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg >
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com >
2026-05-11 10:41:12 -04:00
a51376b3f0
[Performance][DSR1]: Fused RoPE+KVCache+q_concat for MLA ( #40392 )
...
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com >
Signed-off-by: Rohan Potdar <66227218+Rohan138@users.noreply.github.com >
Co-authored-by: ElizaWszola <ewszola@redhat.com >
2026-05-11 14:10:50 +00:00
Noa Neria and GitHub
ac062147fa
Avoid silent weights corruption when loading Nemotron Nano VL with reusable-buffer loaders like runai distributed streaming ( #42244 )
...
Signed-off-by: Noa Neria <nneria@nvidia.com >
2026-05-11 12:03:14 +00:00
Chauncey and GitHub
617239b70c
[Frontend]Responses API supports chat_template_kwargs ( #42272 )
...
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com >
2026-05-11 11:59:39 +00:00
Nicolò Lucchesi and GitHub
5672d100ed
[KV Connector][NIXL][Bugfix] Fix NIXL handshake failures not honoring kv_load_failure_policy ( #40364 )
...
When NIXL handshake fails (e.g., due to compatibility hash mismatch
between prefill and decode instances), requests fail with "engine dead"
error instead of gracefully falling back to local recomputation as configured
by kv_load_failure_policy='recompute'.
Signed-off-by: NickLucche <nlucches@redhat.com >
2026-05-11 09:37:21 +00:00
Nicolò Lucchesi and GitHub
770e9bd6b3
[Nixl][PD] Lease renewal TTL KV blocks on P ( #41383 )
...
Signed-off-by: NickLucche <nlucches@redhat.com >
2026-05-11 09:27:30 +00:00
f9f770ca0b
fix nixl side-channel host selection ( #41806 )
...
Signed-off-by: Shahar Mor <smor@nvidia.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-05-11 07:40:37 +00:00
Jee Jee Li and GitHub
05d610e5cd
[CI/Build] Reduce LoRA model tests. ( #42266 )
...
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai >
2026-05-11 14:49:08 +08:00
wangxiyuan and GitHub
5536fc0c01
[Misc] Replace mamba_type string literals with MambaAttentionBackendEnum ( #41188 )
...
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com >
2026-05-11 03:59:36 +00:00
171019ab19
add fused mhc_post_pre kernel ( #41536 )
...
Signed-off-by: george <george@inferact.ai >
Co-authored-by: george <george@inferact.ai >
2026-05-10 19:56:52 -07:00
Haoqing Wang and GitHub
879a8c3180
Fix Molmo2 image token metadata ( #42162 )
...
Signed-off-by: Haoqi Wang <78337154+hqhq1025@users.noreply.github.com >
2026-05-11 01:19:21 +00:00
1b57eb41f2
[MoE] Move various experts classes to fused_moe/experts/ ( #41979 )
...
Signed-off-by: Jackmin801 <ongjackm@gmail.com >
Signed-off-by: Robert Shaw <robertgshaw2@gmail.com >
Signed-off-by: Jackmin801 <56836461+Jackmin801@users.noreply.github.com >
Signed-off-by: Bill Nell <bnell@redhat.com >
Co-authored-by: Jackmin801 <ongjackm@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Robert Shaw <robertgshaw2@gmail.com >
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
Co-authored-by: Jackmin801 <56836461+Jackmin801@users.noreply.github.com >
2026-05-11 07:54:33 +08:00
Ronen Schaffer and GitHub
e175192d33
[KV Offload] Pass ReqContext to touch(), complete_load(), and complete_store() ( #41366 )
...
Signed-off-by: Ronen Schaffer <ronen.schaffer@ibm.com >
2026-05-10 15:09:25 +03:00
Ethan Feng and GitHub
a2c9d548d7
[Docs] Fix broken local links ( #42160 )
...
Signed-off-by: Ethan Feng <ethan.fengch@gmail.com >
2026-05-10 01:15:38 -07:00
Mohammad Miadh Angkad and GitHub
efd0e7789d
Fix mypy failure on main ( #42197 )
...
Signed-off-by: Mohammad Miadh Angkad <MAngkad.BSDSBA2027@aim.edu >
2026-05-10 07:55:57 +00:00
a5d0a5afba
[Frontend][Bugfix] Abort ASR engine requests on cancellation ( #41266 )
...
Signed-off-by: abdulrahman-cohere <abdulrahman.abdulrazzag@cohere.com >
Signed-off-by: <>
Co-authored-by: Cursor Agent <cursor-agent@cursor.com >
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com >
2026-05-09 23:51:11 -07:00
Andreas Karatzas and GitHub
f2840120f6
[ROCm][CI] Fix NIXL spec-decode acceptance startup and diagnostics ( #41313 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
2026-05-10 14:50:16 +08:00
