 Stan WozniakandGitHub
|
dc66e01a70
|
[Hybrid] Marconi-style admission policy for hybrid cache (#37898)
Signed-off-by: Stanislaw Wozniak <stw@zurich.ibm.com>
|
2026-06-10 10:03:13 -07:00 |
|
|
|
0bae1d3848
|
[MRV2][Spec Decode] DFlash (#44586)
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
Signed-off-by: Benjamin Chislett <chislett.ben@gmail.com>
Co-authored-by: Giancarlo Delfin <gdelfin@inferact.ai>
|
2026-06-10 08:47:46 -07:00 |
|
|
|
af65e08fc5
|
KV-Cache multi-tier offloading async batched lookup (#44193)
Signed-off-by: Effi Ofer <effi.ofer@gmail.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
|
2026-06-10 14:59:30 +00:00 |
|
|
|
9dfc313bdc
|
Feature/offloading manager stats (#35669)
Signed-off-by: Sriusa4414@gmail.com
Signed-off-by: srinivas_oo7 <Sriusa4414@gmail.com>
Signed-off-by: srinivas_oo7 <sklinkedin0120@gmail.com>
Signed-off-by: Srinivasoo7 <158864704+Srinivasoo7@users.noreply.github.com>
Signed-off-by: Or Ozeri <oro@il.ibm.com>
Co-authored-by: srinivas_oo7 <sklinkedin0120@gmail.com>
Co-authored-by: Srinivasoo7 <158864704+Srinivasoo7@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
|
2026-06-10 12:44:55 +00:00 |
|
|
Juan Pérez de AlgabaandGitHub
|
8a5cf1ccd6
|
[Security] Fix remote DoS via invalid recovered token reinjection (#44744)
Signed-off-by: jperezde <jperezde@redhat.com>
|
2026-06-10 02:31:43 -07:00 |
|
|
Andreas KaratzasandGitHub
|
82a42234be
|
[ROCm][CI] Defer AITER sampler import and isolate server test PYTHONPATH (#44823)
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
|
2026-06-10 08:56:11 +00:00 |
|
|
|
c1d754d681
|
[Mooncake] Use all HCAs on multi-NIC hosts instead of GPU-indexed RNIC selection (#43799)
Signed-off-by: Dao Le <Dao007forever@gmail.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
2026-06-09 11:05:36 -07:00 |
|
|
Nicolò LucchesiandGitHub
|
6690a0c4de
|
[PD][Bugfix] Fix KV Cache sharing with HMA (#44629)
Signed-off-by: NickLucche <nlucches@redhat.com>
|
2026-06-09 06:10:06 -07:00 |
|
|
Nicolò LucchesiandGitHub
|
dab60fc658
|
[Bugfix][CI] Fix test_offloading_connector.py::test_fs_tiering_offloading (#44903)
Signed-off-by: NickLucche <nlucches@redhat.com>
|
2026-06-09 00:57:34 -07:00 |
|
|
Andreas KaratzasandGitHub
|
05cb606cad
|
[ROCm][CI] Re-route NixlConnector jobs (#44809)
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
|
2026-06-08 18:57:11 -05:00 |
|
|
Wentao YeandGitHub
|
2c27c294c0
|
[Model Runner V2] Fix mrv2 mm lora issue (#44450)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
|
2026-06-08 14:30:09 -04:00 |
|
|
|
303916e93d
|
[Bugfix]: Fix assertion in MambaManager.allocate_slots() (#39562)
Signed-off-by: Holworth <kangqihan17@mails.ucas.ac.cn>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-06-08 00:34:37 -04:00 |
|
|
Yifan QiaoandGitHub
|
51ef688831
|
[Bugfix][Mooncake] Fix per-group block_size/block_hash and group_idx in MooncakeStoreConnector KV events (#44103)
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
|
2026-06-07 07:12:43 +00:00 |
|
|
Chaojun ZhangandGitHub
|
810966453a
|
[XPU] Support cpu kv offloading and tiering offloading on XPU platform (#36423)
Signed-off-by: Chaojun Zhang <chaojun.zhang@intel.com>
|
2026-06-07 09:59:28 +08:00 |
|
|
Nick HillandGitHub
|
3b3d5287fa
|
[BugFix] Resolve multiple async kv load deadlock (#44560)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
|
2026-06-06 23:05:47 +00:00 |
|
|
Walter Beller-MoralesandGitHub
|
c73b0d0db9
|
[Core][Engine] allow DP ray placement groups to be set on specific nodes (#44669)
Signed-off-by: walterbm <walter.beller.morales@gmail.com>
|
2026-06-05 20:07:47 +00:00 |
|
|
Effi OferandGitHub
|
6a894574bf
|
Add objectstore as a secondary tier to multi-tier kv cache offloading (#41968)
Signed-off-by: Effi Ofer <effi.ofer@gmail.com>
|
2026-06-05 18:05:41 +03:00 |
|
|
|
7fe7800fa4
|
[BUG] Fix FP64 Gumbel precision coverage (#43150)
Signed-off-by: tianyu-z <zhangtianyupro@gmail.com>
Signed-off-by: Tianyu Zhang <53099276+tianyu-z@users.noreply.github.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-06-05 19:04:14 +08:00 |
|
|
Nicolò LucchesiandGitHub
|
d98b8f371c
|
[NixlConnector] Initiate deprecation cycle for kv_both role (#43874)
Signed-off-by: NickLucche <nlucches@redhat.com>
|
2026-06-05 11:08:17 +02:00 |
|
|
qizixiandGitHub
|
96229fa99e
|
[KVConnector][1/N] PP-aware handshake aggregation and intermediate-PP output plumbing (#43720)
Signed-off-by: zixi-qi <zixi@inferact.ai>
|
2026-06-04 22:04:19 -07:00 |
|
|
Nicolò LucchesiandGitHub
|
68f5e565c9
|
[PD][Nixl] Mamba prefix caching mode support (#42554)
Signed-off-by: NickLucche <nlucches@redhat.com>
|
2026-06-04 06:41:46 -07:00 |
|
|
|
90619351e3
|
[Attention] Mamba attention module refactor - LINEAR (#43556)
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-06-04 18:45:29 +08:00 |
|
|
|
a6183563b6
|
[Prefix Caching] DeepSeekv4 - Support selective prefix-cache retention for sliding-window KV cache (#43447)
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
|
2026-06-04 00:48:31 -07:00 |
|
|
|
e6018c644a
|
[Refactor] Remove dead code in tests and parallel_state (#41471)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-06-03 19:32:39 -07:00 |
|
|
maobaolongandGitHub
|
b58e082d95
|
[KV Connector] Update lmcache kv_offloading_backend to use LMCacheMPConnector (#42865)
Signed-off-by: baoloongmao <baoloongmao@tencent.com>
|
2026-06-03 19:23:55 -07:00 |
|
|
|
0414d75410
|
[XPU] skip unapplied UT in test_gpu_model_runner.py (#44289)
Signed-off-by: Yan Ma <yan.ma@intel.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-06-04 08:48:17 +08:00 |
|
|
Wentao YeandGitHub
|
dad95e34d8
|
[Feature] Support batch invariant rms norm with residual (#42453)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
|
2026-06-03 15:22:01 -04:00 |
|
|
|
0c6631f02a
|
[KVCache] Support Pluggable KVCacheSpec (#37505)
Signed-off-by: MengqingCao <cmq0113@163.com>
Signed-off-by: Mengqing Cao <cmq0113@163.com>
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-06-03 09:05:16 -07:00 |
|
|
Nicolò LucchesiandGitHub
|
df7252c343
|
[CI] Align PD tests to HMA on by default (#44174)
Signed-off-by: NickLucche <nlucches@redhat.com>
|
2026-06-04 00:04:30 +08:00 |
|
|
|
3d76f395e3
|
[SharedOffloadRegion] Align blocks to page-size (#43689)
Signed-off-by: varun sundar rabindranath <vsundarr@redhat.com>
Co-authored-by: varun sundar rabindranath <vsundarr@redhat.com>
|
2026-06-03 14:25:57 +03:00 |
|
|
Andy LoandGitHub
|
95b1615ec9
|
[Perf] Improve multimodal item handling from O(n) to O(log n) per step (#44212)
Signed-off-by: Andy Lo <andy@mistral.ai>
|
2026-06-03 11:00:26 +00:00 |
|
|
|
1fa9ea09f6
|
[Perf] Triton fast path for small CPU→GPU swap_blocks_batch in the offloading connector (#42212)
Signed-off-by: Itay Etelis <itay.etelis@ibm.com>
Co-authored-by: Itay Etelis <itay.etelis@ibm.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-06-03 13:38:17 +03:00 |
|
|
Rotem ShavittandGitHub
|
3f0a91bb96
|
Nit Changes in Tiered KV Offload (#44293)
Signed-off-by: Rotem Shavitt <rshavitt@gmail.com>
|
2026-06-02 21:53:21 -07:00 |
|
|
Andreas KaratzasandGitHub
|
53b88d1dfc
|
[CI] Reject out-of-vocabulary before they reach the GPU logprob path (#44042)
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
|
2026-06-02 22:27:52 -05:00 |
|
|
Nick HillandGitHub
|
e15f20258b
|
[ModelRunnerV2] Avoid pipeline parallel bubbles (#42187)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
|
2026-06-02 14:02:01 -07:00 |
|
|
Yifan QiaoandGitHub
|
e9e08c49b9
|
[Bugfix] Cache the EAGLE/MTP lookahead block in the SWA prefix-cache mask (#44082)
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
|
2026-06-02 12:21:07 -07:00 |
|
|
|
da107a59e5
|
[MRV2] Also enable MRV2 for Llama and Mistral dense models (#43458)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: yewentao256 <zhyanwentao@126.com>
|
2026-06-02 11:18:46 -07:00 |
|
|
Nick HillandGitHub
|
cab5c9a2a9
|
[Core] Move max_concurrent_batches to VllmConfig (#44274)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
|
2026-06-02 08:57:25 -07:00 |
|
|
|
654bd2bca4
|
[Bugfix] Sync block_size from EngineCore to frontend for hybrid Mamba… (#42967)
Signed-off-by: Amit Gruner <agruner@crusoe.ai>
Co-authored-by: Amit Gruner <agruner@crusoe.ai>
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com>
|
2026-06-02 13:41:00 +00:00 |
|
|
Shreyas KulkarniandGitHub
|
0eeba5eec1
|
Fix DFlash prefix cache corruption due to missing lookahead block (#42971)
Signed-off-by: Shreyas Kulkarni <shreyas.gp269@gmail.com>
|
2026-06-02 12:06:33 +00:00 |
|
|
Ronen SchafferandGitHub
|
2a2b5ca791
|
[KV Offload] Add on_schedule_end() hook to separate step lifecycle from event draining (#44206)
Signed-off-by: Ronen Schaffer <ronen.schaffer@ibm.com>
|
2026-06-02 13:42:52 +03:00 |
|
|
|
d247a9dc13
|
[EC Connector] Non blocking EC Connector lookup (#41627)
Signed-off-by: omerpaz95 <omerpaz95@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-06-02 08:48:25 +00:00 |
|
|
Yifan QiaoandGitHub
|
7c37096620
|
[Core][Refactor]: thread scheduler_block_size into KVCacheManager and KVCacheCoordinator (#44165)
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
|
2026-06-02 01:14:44 -07:00 |
|
|
JooHo LeeandGitHub
|
a045c7425f
|
[MM][CG] Profile encoder CUDA graph pool memory (#41714)
Signed-off-by: JooHo Lee <jooho414@gmail.com>
|
2026-06-02 12:27:34 +08:00 |
|
|
Or OzeriandGitHub
|
480fadab1b
|
[BugFix][kv_offload]: Prevent offloading stale sliding window blocks (#42959)
Signed-off-by: Or Ozeri <oro@il.ibm.com>
|
2026-06-02 05:59:48 +03:00 |
|
|
|
d68f0b220e
|
[Bugfix][Mooncake] Release GPU pin on failed store in MooncakeStoreConnector (#43742)
Signed-off-by: Dao Le <Dao007forever@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-06-01 18:29:18 -07:00 |
|
|
Nick HillandGitHub
|
e4cbc4385d
|
[Test][BugFix] Fix double-BOS in PD+specdec acceptance test (#44234)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
|
2026-06-01 14:31:12 -07:00 |
|
|
|
266b9d9c64
|
[Frontend][Core] Add sparse NCCL weight transfer support for in-place updates (#40096)
Signed-off-by: Siddharth Bedekar <bedeksid@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-06-01 15:37:30 -04:00 |
|
|
|
fd9e91d7e4
|
[ROCm][CI] Fix and stabilize EAGLE3 acceptance tests (#41294)
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Signed-off-by: Micah Williamson <micah.williamson@amd.com>
Co-authored-by: Micah Williamson <micah.williamson@amd.com>
|
2026-06-01 12:40:01 -05:00 |
|
|
Nicolò LucchesiandGitHub
|
7ebc0ec104
|
[CI] Nixl+SimpleCPUOffloadingConnector unit tests (#43871)
Signed-off-by: NickLucche <nlucches@redhat.com>
|
2026-05-29 02:40:42 -07:00 |
|