karylab_agents/vllm
1
0
Fork 0
forked from Karylab-cklius/vllm
Code Pull Requests 2 Actions 1 Packages Activity
17,191 Commits 354 Branches 175 Tags
dependabot/github_actions/actions/setup-python-6.2.0
Commit Graph
17191 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Nicolò LucchesiandGitHub 5672d100ed [KV Connector][NIXL][Bugfix] Fix NIXL handshake failures not honoring kv_load_failure_policy (#40364) 
When NIXL handshake fails (e.g., due to compatibility hash mismatch
between prefill and decode instances), requests fail with "engine dead"
error instead of gracefully falling back to local recomputation as configured
by kv_load_failure_policy='recompute'.

Signed-off-by: NickLucche <nlucches@redhat.com>
 2026-05-11 09:37:21 +00:00
Nicolò LucchesiandGitHub 770e9bd6b3 [Nixl][PD] Lease renewal TTL KV blocks on P (#41383) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2026-05-11 09:27:30 +00:00
Cyrus LeungandGitHub 9efdddca28 [Model] Fix missing maybe_prefix (#42280) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-05-11 09:04:06 +00:00
QiuandGitHub b1b59720b2 bugfix(flashinfer,dcp): remove kv_cache_layout for BatchDCPPrefillWrapper._new_tokens. (#38895) 
Signed-off-by: QiuChunshuo <qiuchunshuo@huawei.com>
 2026-05-11 08:11:49 +00:00
shaharmor98GitHubmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		f9f770ca0b fix nixl side-channel host selection (#41806) 
Signed-off-by: Shahar Mor <smor@nvidia.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-05-11 07:40:37 +00:00
Haoqing WangandGitHub 5cba6839e6 Document MolmoWeb hf_overrides (#42163) 
Signed-off-by: Haoqi Wang <78337154+hqhq1025@users.noreply.github.com>
 2026-05-10 23:58:22 -07:00
Jee Jee LiandGitHub 05d610e5cd [CI/Build] Reduce LoRA model tests. (#42266) 
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
 2026-05-11 14:49:08 +08:00
汪志鹏GitHubWang, Zhipeng | RASIA <zhipeng.wang@rakuten.com>CursorChauncey
 		581b5e9afc [Frontend] Return rendered prompt text in chat completion response (#42052) 
Signed-off-by: Wang, Zhipeng | RASIA <zhipeng.wang@rakuten.com>
Co-authored-by: Wang, Zhipeng | RASIA <zhipeng.wang@rakuten.com>
Co-authored-by: Cursor <cursor@cursor.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
 2026-05-11 13:53:39 +08:00
wangxiyuanandGitHub 5536fc0c01 [Misc] Replace mamba_type string literals with MambaAttentionBackendEnum (#41188) 
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
 2026-05-11 03:59:36 +00:00
vllmellmandGitHub 7f95e66a11 [ROCm][Bugfix]: dynamically align BLOCK_DMODEL with Lv in MLA decode kernel (#41119) 
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
 2026-05-11 11:14:19 +08:00
yzong-rhandGitHub b1687527b8 [Bugfix] Gemma 4 chat template crash with missing tool name and tool id (#42188) 
Signed-off-by: Yifan <yzong@redhat.com>
 2026-05-11 03:07:45 +00:00
gnovackGitHubgeorge
 		171019ab19 add fused mhc_post_pre kernel (#41536) 
Signed-off-by: george <george@inferact.ai>
Co-authored-by: george <george@inferact.ai>
 2026-05-10 19:56:52 -07:00
Haoqing WangandGitHub 879a8c3180 Fix Molmo2 image token metadata (#42162) 
Signed-off-by: Haoqi Wang <78337154+hqhq1025@users.noreply.github.com>
 2026-05-11 01:19:21 +00:00
bnellnmGitHubJackmin801ClaudeRobert ShawRobert ShawJackmin801
 		1b57eb41f2 [MoE] Move various experts classes to fused_moe/experts/ (#41979) 
Signed-off-by: Jackmin801 <ongjackm@gmail.com>
Signed-off-by: Robert Shaw <robertgshaw2@gmail.com>
Signed-off-by: Jackmin801 <56836461+Jackmin801@users.noreply.github.com>
Signed-off-by: Bill Nell <bnell@redhat.com>
Co-authored-by: Jackmin801 <ongjackm@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Robert Shaw <robertgshaw2@gmail.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Jackmin801 <56836461+Jackmin801@users.noreply.github.com>
 2026-05-11 07:54:33 +08:00
Mohammad Miadh AngkadandGitHub 21943d4c25 [Performance] Make safetensors checkpoint prefetch settings configurable (#41499) 
Signed-off-by: Mohammad Miadh Angkad <MAngkad.BSDSBA2027@aim.edu>
 2026-05-10 15:55:15 +00:00
Isotr0pyGitHubqizixi
 		f396bee56f [DSV4] Add PP support for deepseek-v4 (#41694) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: qizixi <22851944+zixi-qi@users.noreply.github.com>
 2026-05-10 15:47:26 +00:00
VensenGitHubmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		215e2f7990 [Bugfix][Mamba] IMA in causal_conv1d kernel for long sequences (#41617) 
Signed-off-by: vensen <vensenmu@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-05-10 12:38:28 +00:00
Ronen SchafferandGitHub e175192d33 [KV Offload] Pass ReqContext to touch(), complete_load(), and complete_store() (#41366) 
Signed-off-by: Ronen Schaffer <ronen.schaffer@ibm.com>
 2026-05-10 15:09:25 +03:00
Jonathan MamouGitHubBenjamin Chislett
 		a54f0d1049 [CPU] Fix spec decode kernel signatures for synthetic mode compatibility (#41932) 
Signed-off-by: jmamou <jonathan.mamou@intel.com>
Signed-off-by: Jonathan Mamou <jonathan.mamou@intel.com>
Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com>
 2026-05-10 12:07:15 +00:00
Isotr0pyandGitHub 48698b1b9b [Bugfix] Fuse Qwen3.5 in_qkvz_proj forwarding with LoRA enabled (#37912) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
 2026-05-10 10:59:02 +00:00
Andreas KaratzasandGitHub 0a309b5ee9 [ROCm] Cap Triton paged attention block size to fix ROCm shared memory OOM (#38502) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-05-10 10:03:00 +00:00
Jee Jee LiandGitHub 84f7a55340 [CI] Trigger LoRA test when changing MoE code. (#42196) 
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
 2026-05-10 01:26:09 -07:00
Ethan FengandGitHub a2c9d548d7 [Docs] Fix broken local links (#42160) 
Signed-off-by: Ethan Feng <ethan.fengch@gmail.com>
 2026-05-10 01:15:38 -07:00
Yongye ZhuandGitHub 301305c093 Add @zyongye to CODEOWNERS (#42200) 
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
 2026-05-10 16:07:32 +08:00
Mohammad Miadh AngkadandGitHub efd0e7789d Fix mypy failure on main (#42197) 
Signed-off-by: Mohammad Miadh Angkad <MAngkad.BSDSBA2027@aim.edu>
 2026-05-10 07:55:57 +00:00
abdulrahman-cohereGitHubCursor AgentNicolò Lucchesi
 		a5d0a5afba [Frontend][Bugfix] Abort ASR engine requests on cancellation (#41266) 
Signed-off-by: abdulrahman-cohere <abdulrahman.abdulrazzag@cohere.com>
Signed-off-by: <>
Co-authored-by: Cursor Agent <cursor-agent@cursor.com>
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com>
 2026-05-09 23:51:11 -07:00
Andreas KaratzasandGitHub f2840120f6 [ROCm][CI] Fix NIXL spec-decode acceptance startup and diagnostics (#41313) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-05-10 14:50:16 +08:00
Dao007foreverandGitHub 3f5bd482f5 [Bugfix][KV Transfer][NIXL] Notify P node on pre-admission rejection to free stranded KV blocks (#41269) 2026-05-09 22:52:09 -07:00
Andreas KaratzasandGitHub fb1ac806c5 [ROCm][CI] Stabilize ROCm shutdown and distributed compile CI (#41573) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-05-10 03:47:40 +00:00
Wei ZhaoandGitHub 986edc858a [Bugfix] Fix DeepSeek v4 topk numerical issue for unaligned max-model-len (#42169) 2026-05-09 20:30:08 -07:00
Abhishek GuptaGitHubAbhiOnGithub
 		27d3bac272 docs: clarify Gemma 4 assistant speculative decoding (#42180) 
Signed-off-by: AbhiOnGithub <abhiOnGithub@users.noreply.github.com>
Co-authored-by: AbhiOnGithub <abhiOnGithub@users.noreply.github.com>
 2026-05-09 20:08:44 -07:00
Itay EtelisGitHubItay EtelisOr OzeriItay Etelismergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		00b0618a03 Use CU_MEMCPY_SRC_ACCESS_ORDER_ANY for batch KV cache swaps (#39306) 
Signed-off-by: Itay Etelis <itay.etelis@ibm.com>
Signed-off-by: Itay Etelis <etelis2019@gmail.com>
Signed-off-by: Itay Etelis <92247226+Etelis@users.noreply.github.com>
Co-authored-by: Itay Etelis <itay.etelis@ibm.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
Co-authored-by: Itay Etelis <etelis2019@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-05-10 05:57:09 +03:00
Christian VanGitHubChristian Van
 		0d382ecde8 Handle optional bool-or-string CLI args in get_kwargs (#40951) 
Signed-off-by: Christian Van <cvan20191@gmail.com>
Co-authored-by: Christian Van <cvan20191@gmail.com>
 2026-05-09 19:47:21 -07:00
Isotr0pyandGitHub 1029e5ef28 [CI/Build] Use modelscope's international site for regression test (#42176) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-05-09 19:47:09 -07:00
Wang XingranGitHubHongjian Zhang
 		0b272a6e01 [Bugfix] Fix SP pass for multimodal models and PP+SP residual handling (#33322) 
Signed-off-by: Xingran Wang <wangxingran123456@outlook.com>
Signed-off-by: Hongjian Zhang <hirokenovo@gmail.com>
Co-authored-by: Hongjian Zhang <hirokenovo@gmail.com>
 2026-05-09 19:44:16 -07:00
Nave AssafGitHubmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		dcb3135af7 Fix: Nemotron 3 rescue whitespace-only final_content, not just None (#41846) 
Signed-off-by: Nave Assaf <nassaf@nvidia.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-05-10 02:07:58 +00:00
baonudesifeizhaiGitHubmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		bc5fdc1e6a Add NVFP4 all-gather GEMM fusion for AsyncTP (#41882) 
Signed-off-by: roG0d <baonudesifeizhai@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-05-10 01:13:22 +00:00
aoshen02andGitHub 006af4b956 [Bugfix] Skip routed-experts hot path when disabled (#42148) 2026-05-09 18:01:04 -07:00
Wentao YeandGitHub ea0e501bb1 [KV Connector] Remove compat support for pre-v0.12.0 constructor signatures without KVCacheConfig (#39832) 
The v0.12.0 release contained initial support for HMA in KV Connectors. As part
of these changes, a KVCacheConfig argument was added to KV connector
constructors. Backwards compatibility support for out-of-tree connectors was
included in this change, with a very prominent warning. See #25712 and #27887.

Since the warning has been around for over 5 months, we can safely remove
the support of it.

Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-05-09 23:39:46 +00:00
Wentao YeandGitHub f80aa53c9d [Refactor] Nixl util using lazy init (#41392) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-05-09 17:46:52 -04:00
Juhi MittalandGitHub 7a2b596982 [Quantization] Add ModelOpt NVFP4 W4A16 (4-bit weights, fp16/bf16 activations) support (#41769) 
Signed-off-by: Juhi Mittal <juhim@nvidia.com>
 2026-05-09 21:15:50 +00:00
Jiangyun ZhuandGitHub 2ee8c2a56e [SpecDecoding] extend mtp support for mimo 2.5 (#41905) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
 2026-05-09 18:22:59 +00:00
SoluMilkenandGitHub cd74911d92 [Model] use AutoWeightsLoader for DeepSeekV2 (#41706) 
Signed-off-by: SoluMilken <ypiheyn.imm02g@g2.nctu.edu.tw>
 2026-05-10 01:55:25 +08:00
SoluMilkenandGitHub 25abddc1a5 [BugFix] Fix Gemma4 'layers.0.moe.experts.0.down_proj_packed' KeyError issue (#40708) 
Signed-off-by: SoluMilken <ypiheyn.imm02g@g2.nctu.edu.tw>
 2026-05-09 17:20:44 +00:00
Nicolò LucchesiGitHubZhanqiuHu
 		171d59ae8d [Bugfix][PD] Fix DSv4 Disaggregated (#41957) 
Signed-off-by: NickLucche <nlucches@redhat.com>
Co-authored-by: ZhanqiuHu <zhu@redhat.com>
 2026-05-09 16:48:24 +00:00
Thomas ParnellGitHubJiangyun Zhu
 		3dda9aeb54 [Bugfix] Remove nested torch.compile in GDN rearrange_mixed_qkv causing CUDA graph capture failure (#42070) 
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com>
 2026-05-09 08:30:55 -07:00
KermitandGitHub adb6d96516 [Bugfix] Fix GDN KKT precision loss on Hopper GPUs by aligning tl.dot operand layout with WGMMA (#42076) 
Signed-off-by: kermit <ckeming@outlook.com>
 2026-05-09 13:08:46 +00:00
Thien TranandGitHub 530d371302 [DSv4] Improved fused Indexer Q quant kernel (#41428) 2026-05-09 01:20:32 -07:00
Micah WilliamsonandGitHub 34ab4f2565 [ROCm] Upgrade aiter to v0.1.13-rc5 (#42113) 
Signed-off-by: Micah Williamson <micah.williamson@amd.com>
 2026-05-09 08:13:45 +00:00
Jee Jee LiandGitHub ecd0b60aad [LoRA] Initial EP support for LoRA (#40867) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
 2026-05-09 00:31:23 -07:00