karylab_agents/vllm
1
0
Fork 0
forked from Karylab-cklius/vllm
Code Pull Requests 2 Actions 1 Packages Activity
18,963 Commits 354 Branches 175 Tags
bz/rust-code-coverage
Commit Graph
153 Commits
Author SHA1 Message Date
aoshen02andGitHub 9459fc6471 [Bugfix][RL] Set vLLM config during weight reload (#45989) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
 2026-07-20 15:02:56 +08:00
devalshahamdGitHubDeval Shah
 		fcd2255d16 [Hardware][GPU] Profiler config additional to increase it scope and annotation details (#37524) 
Signed-off-by: devalshahamd <deval.shah@amd.com>
Signed-off-by: Deval Shah <devashah@amd.com>
Signed-off-by: Deval Shah <deval.shah@amd.com>
Co-authored-by: Deval Shah <devashah@amd.com>
 2026-07-17 13:38:59 -07:00
wendadawenGitHubTianyu Guo
 		69d4f5ef63 [Bugfix][Multimodal] Fix Qwen3-Omni use_audio_in_video with mixed image/video inputs (#46213) 
Signed-off-by: wendadawen <wendadawen@qq.com>
Signed-off-by: Tianyu Guo <guoty9@mail2.sysu.edu.cn>
Co-authored-by: Tianyu Guo <guoty9@mail2.sysu.edu.cn>
 2026-07-17 08:31:16 +00:00
vx120GitHubcrp0128aoshen02Claude Opus 4.8
 		fc1c548093 Runtime Draft Weight Update for Speculative Decoding (#46725) 
Signed-off-by: vx120 <893600387@qq.com>
Signed-off-by: vx120 <57470515+vx120@users.noreply.github.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: crp0128 <191679376@qq.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
 2026-07-11 22:51:53 -07:00
Lucas WilkinsonGitHubClaudeOpenAI CodexMatthew BonanniNicolò Lucchesi
 		51878e5b6e [2/N][KV-Cache Layout Refactor] Pack K/V into the content dim across attention backends (#44455) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com>
 2026-07-11 11:11:16 -04:00
Yan XuGitHubJingyi Yang
 		95ed0feaa5 DCP supports hybrid attention (#40996) 
Signed-off-by: YanXu <yancey.yx@alibaba-inc.com>
Signed-off-by: Jingyi Yang <girasoleyang@gmail.com>
Co-authored-by: Jingyi Yang <girasoleyang@gmail.com>
 2026-07-09 21:34:45 -07:00
Woosuk KwonGitHubClaude Fable 5Nick Hill
 		85b3a7264b [Bugfix][Model Runner V2] Order uniform decodes first so spec decodes aren't misclassified as prefills (#47381) 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
 2026-07-09 14:26:27 +01:00
Benjamin ChislettGitHubWentao Ye
 		1cd75b3dd4 [Bugfix] Fix race condition in KVBlockZeroer (#48085) 
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
Signed-off-by: Benjamin Chislett <chislett.ben@gmail.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2026-07-09 09:18:19 +00:00
Nick HillandGitHub a5d19cbb95 [Core] Move MRV1 late_interaction_runner.py out of MRV2 subtree (#48014) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-07-08 18:30:11 +00:00
Walter Beller-MoralesandGitHub dd127d82ed [Core][Engine] only materialize tokens when thinking budget is in req (#47053) 
Signed-off-by: walterbm <walter.beller.morales@gmail.com>
 2026-07-07 21:02:38 -06:00
Nils MattesonGitHubNils Mattesonmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		e97c3cb303 [Core] Persist and reuse the memory-profiling result across boots (opt-in) (#47388) 
Signed-off-by: Nils Matteson <nils@thaw.sh>
Signed-off-by: Nils Matteson <nilsmatteson@icloud.com>
Co-authored-by: Nils Matteson <nils@thaw.sh>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-07-08 00:53:02 +00:00
Łukasz ŚlusarczykGitHubmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Kunshang Ji
 		78a04c208d [XPU] Fix CUDA API shims breaking Torch Dynamo during AOT compile (#43092) 
Signed-off-by: Łukasz Ślusarczyk <lukasz.slusarczyk@intel.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-07-06 08:29:20 +08:00
Woosuk KwonGitHubClaude Fable 5
 		a47f38f825 [Bugfix][Model Runner V2][Spec Decode] Fix int32 offset overflow in block verification kernels (#47383) 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-07-02 07:32:38 -07:00
Yiliu DongGitHubNick Hill
 		00eb7cefa3 [Bugfix] Prevent padding placeholders from reaching embeddings (#47029) 
Signed-off-by: qianlihuang <91178480+qianlihuang@users.noreply.github.com>
Signed-off-by: Yiliu Dong <91178480+qianlihuang@users.noreply.github.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
 2026-07-01 09:26:03 -07:00
Aaron HaoGitHubClaude Sonnet 4.6
 		77a9c5ae28 Weight sync refactor + move sparse nccl engine (#44353) 
Signed-off-by: hao-aaron <ahao@anyscale.com>
Signed-off-by: haoaaron <ahao@anyscale.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-07-01 01:25:19 -07:00
Nils MattesonandGitHub a461070d1c [Core] Make sleep-mode backend capability flags communicator-agnostic (#47243) 2026-07-01 07:17:44 +00:00
Nils MattesonandGitHub 93d8f834dd [Core] Pluggable sleep-mode backend abstraction (RFC #34303) (#44074) 2026-06-30 22:00:53 -07:00
Nick HillGitHubziminghuang
 		fcaa84efa7 [BugFix] Gate MRV2 mixed sparse-MLA warmup on max_num_seqs > 1 (#47050) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: ziminghuang <ziminghuang@inferact.ai>
 2026-06-30 16:31:27 +01:00
ChaunceyandGitHub 11a12305c0 [Model Runner V2][Spec Decode] Handle tuple hidden states from MTP draft models (#46786) 2026-06-27 18:38:07 -07:00
Nick HillandGitHub c6dd32a810 [ModelRunner V2] Support realtime embeddings (#46762) 2026-06-26 19:42:27 -07:00
Brandon PelfreyGitHubRoger Wang
 		af16446bf3 Vram semaphore infra (#44465) 
Signed-off-by: Brandon Pelfrey <bpelfrey@nvidia.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
 2026-06-26 17:32:51 -07:00
Nick HillGitHubClaude Opus 4.8Roger Wang
 		3e6529cc0e [Bugfix][Spec Decode] Fix EAGLE drafter multimodal encoder cache misses (#46315) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
 2026-06-22 18:14:02 +00:00
Alex SteinerGitHubClaude Opus 4.8Giancarlo DelfinYifan Qiao
 		2cac89f9da [Spec Decode] Support mixed KV page sizes for DFlash (#45181) 
Signed-off-by: Alex Steiner <asteiner@nvidia.com>
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Giancarlo Delfin <gdelfin@inferact.ai>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
 2026-06-21 22:45:14 +08:00
Nick HillandGitHub 7df3d7dada [Core] Ensure memory is pinned prior to async h2d copy (#45424) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-06-20 20:02:24 -07:00
Woosuk KwonandGitHub 16908e132e [MRV2] Make FP32 Gumbel sampling more accurate (#45996) 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
 2026-06-18 19:42:09 +00:00
sungsoo haGitHubThomas Parnell
 		4bf699d310 [Kernel] Support DS Mamba tail copy for MTP align mode (#45473) 
Signed-off-by: Sungsoo Ha <sungsooh@nvidia.com>
Co-authored-by: Thomas Parnell <tom.parnell@gmail.com>
 2026-06-16 22:50:30 +00:00
Song ZhixinGitHubNick Hill
 		8e27a9c215 [PERF] Fuse multi-group block table staged writes (#44944) 
Signed-off-by: jesse <szxfml@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
 2026-06-16 10:53:27 -07:00
Juan Pérez de AlgabaandGitHub 470229c37e [Security] Fix DoS via prompt_embeds on M-RoPE models (#45252) 
Signed-off-by: jperezde <jperezde@redhat.com>
 2026-06-13 10:17:38 +00:00
+1
Lucas WilkinsonGitHubMartin KuklaMatthew BonanniDipika SikkaNickLucchejiahancAlec KohlhoffPorras HuangClaudescoootscooob
 		eb28452b10 [Model] Add DiffusionGemma Support (#45163) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Martin Kukla <martin.kukla@cantab.net>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Dipika Sikka <dsikka@redhat.com>
Co-authored-by: NickLucche <nlucches@redhat.com>
Co-authored-by: jiahanc <173873397+jiahanc@users.noreply.github.com>
Co-authored-by: Alec Kohlhoff <134344302+aleckohlhoff@users.noreply.github.com>
Co-authored-by: Porras Huang <20535584+porrashuang@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: scoootscooob <167050519+scoootscooob@users.noreply.github.com>
 2026-06-11 22:17:35 -07:00
Wentao YeandGitHub 2c27c294c0 [Model Runner V2] Fix mrv2 mm lora issue (#44450) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-06-08 14:30:09 -04:00
Yan MaGitHubmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		0414d75410 [XPU] skip unapplied UT in test_gpu_model_runner.py (#44289) 
Signed-off-by: Yan Ma <yan.ma@intel.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-06-04 08:48:17 +08:00
Nick HillandGitHub e15f20258b [ModelRunnerV2] Avoid pipeline parallel bubbles (#42187) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-06-02 14:02:01 -07:00
Siddharth BedekarGitHubOpenAI Codex
 		266b9d9c64 [Frontend][Core] Add sparse NCCL weight transfer support for in-place updates (#40096) 
Signed-off-by: Siddharth Bedekar <bedeksid@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-06-01 15:37:30 -04:00
Nick HillGitHubmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		7e53283b1c [Core] Cleanup KVConnector handling with PP + fix MRV2 (#43732) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-05-28 13:12:03 -07:00
Nick HillGitHubWentao Ye
 		8c94938cfb [MRV2][BugFix] Fix KV connector handling in spec decode case (#43719) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2026-05-27 06:37:56 +00:00
Lucas WilkinsonGitHubMatthew BonanniNicolò Lucchesi
 		7e33081cee [Attention] Make FlexAttention and FlashAttention use num-blocks first layouts (#42095) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com>
 2026-05-26 19:55:56 -07:00
Francesco FuscoGitHubmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		b730c46352 [Perf] [Hybrid] Fused Triton kernel for GPU-side Mamba state postprocessing (#40172) 
Signed-off-by: Francesco Fusco <ffu@zurich.ibm.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-05-21 04:50:54 -07:00
Siddharth BedekarandGitHub f51f6844f9 [Bugfix][Spec Decode] Wire draft_probs into probabilistic draft_model rejection (#40269) 2026-05-13 21:04:03 -04:00
Jing WangGitHubCopilotWentao Yemergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		27702f6d08 [Bugfix] Fix token loss in PP mode which causes degraded accuracy (#41133) 
Signed-off-by: Jing Wang <jingwang96@qq.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-05-06 14:07:32 -04:00
wi-adamandGitHub b53c507bc9 [Bugfix] Skip PP sampled-token receive on last rank during async scheduling (#40749) 
Signed-off-by: Adam Winstanley <adam@winstanley.industries>
 2026-05-06 05:31:14 +00:00
wliao2andGitHub 32e0c0bfa2 refactor hard coded device string in test files under tests/v1 and tests/lora (#37566) 
Signed-off-by: Liao, Wei <wei.liao@intel.com>
 2026-04-03 11:21:47 +08:00
wang.yuqiandGitHub a9b4f07ba2 [Frontend] Re-enable running MaxSim on GPU (#38620) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-04-03 00:03:13 +08:00
Chendi.XueGitHubMatthew BonanniNicolò Lucchesi
 		3b1dbaad4e [HMA]Fix corner case when hybrid page_size can not be evenly divided issue (blk_size=64,tp=4) (#37467) 
Signed-off-by: Chendi Xue <chendi.xue@intel.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Signed-off-by: Chendi.Xue <chendi.xue@intel.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com>
 2026-03-30 16:47:30 +00:00
Collin McCarthyGitHubNetanel Haber
 		1031c84c36 Fix ambiguous num_blocks for hybrid attn mamba (#37236) 
Signed-off-by: Collin McCarthy <cmccarthy@nvidia.com>
Signed-off-by: Netanel Haber <58652339+netanel-haber@users.noreply.github.com>
Co-authored-by: Netanel Haber <58652339+netanel-haber@users.noreply.github.com>
 2026-03-30 11:09:45 +00:00
Nicolò LucchesiandGitHub cc06b4e86b [Mamba][Bugfix] Raise on insufficient cache blocks instead of silently capping cudagraph sizes (#38270) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2026-03-30 09:41:50 +00:00
Wentao YeandGitHub 995dea1354 [Perf] Remove redundant device copies for CPU-only pooling token IDs, 48.9% E2E throughput improvement (#38139) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-03-29 18:12:50 +00:00
Wentao YeGitHubWoosuk Kwon
 		d7e93e13fb [Feature] EPLB Support for GPU Model Runner v2 (#37488) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>
 2026-03-25 08:16:39 -07:00
Wentao YeandGitHub c59a132f96 [V0 Deprecation] Refactor kv cache from list to element (#37487) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-03-23 20:10:11 -07:00
Andreas KaratzasandGitHub 66f927f205 [Bugfix] Fix pooling non-determinism from pinned prompt_lens aliasing (#37775) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-03-22 03:22:24 +00:00
Francesco FuscoandGitHub 298e510848 [Hybrid] calling get_mamba_groups() once at MambaCopyBuffers.create() (#37318) 
Signed-off-by: Francesco Fusco <ffu@zurich.ibm.com>
 2026-03-21 09:29:43 +00:00