karylab_agents/vllm
1
0
Fork 0
forked from Karylab-cklius/vllm
Code Pull Requests 2 Actions 1 Packages Activity
16,338 Commits 354 Branches 175 Tags
97cd2c41ade8a9c9d85e40299ef4e3bbdddc7903
Commit Graph
887 Commits
Author SHA1 Message Date
Yongye ZhuandClaude Opus 4.7 0012818287 [Attention][TokenSpeed MLA] Fix trtllm LSE parity test: log2 → natural log 
trtllm_ragged_attention_deepseek returns LSE in log2; tokenspeed and
merge_attn_states use natural log. Multiply the trtllm reference by ln 2
before comparison.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
 2026-05-06 02:49:09 +00:00
Yongye ZhuandClaude Opus 4.7 d0e6514bf8 [Attention] Add TOKENSPEED_MLA backend for DeepSeek R1 prefill + decode on Blackwell 
Wires the tokenspeed_mla CuTe DSL kernels into vLLM as a new MLA backend,
covering both prefill (tokenspeed_mla_prefill) and decode
(tokenspeed_mla_decode). Targets Blackwell (SM100) with FP8 KV cache and
DeepSeek R1 MLA dimensions; users opt in via -ac
'{"backend":"TOKENSPEED_MLA","mla_prefill_backend":"TOKENSPEED_MLA"}'.
Includes numeric parity tests against the trtllm reference kernels.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
 2026-05-06 02:49:09 +00:00
Hexiang WangGitHubganyitjtanaatjtanaavllm
 		628c436301 [New Model][ROCm] Add AMD support for DeepSeek V4 (#40871) 
Signed-off-by: ganyi <ygan@amd.com>
Signed-off-by: whx-sjtu <xiaowang990929@gmail.com>
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Signed-off-by: tjtanaavllm <tunjian.tan@amd.com>
Co-authored-by: ganyi <ygan@amd.com>
Co-authored-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: tjtanaavllm <tunjian.tan@amd.com>
 2026-05-05 08:55:37 -07:00
Bowen BaoandGitHub 1e9500410a [ROCm][Quantization][2/N] Refactor quark_moe w4a8 w/ oracle (#39136) 
Signed-off-by: Bowen Bao <bowenbao@amd.com>
 2026-05-04 19:50:38 -07:00
Luka GovedičandGitHub d58c42e19c [vLLM IR] 2/N fused_add_rms_norm and maybe_inplace overload (#36823) 
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Signed-off-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
 2026-05-01 23:41:15 -04:00
sychen52andGitHub 947138b6c2 Add nvfp4 kv cache support (#40177) 
Signed-off-by: Shiyang Chen <shiychen@nvidia.com>
 2026-05-01 04:55:16 +00:00
Juhi MittalGitHubmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		6b6ac6c3c7 [Kernel][MoE] Support GELU on TRT-LLM NvFP4 fused MoE for Gemma4 (#41050) 
Signed-off-by: Juhi Mittal <juhim@nvidia.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-05-01 03:37:43 +00:00
Yongye ZhuGitHubRoger Wang
 		a3c83ff2fd Faster per-token fp8 group quant packed kernel for blackwell (#41326) 
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
 2026-04-30 18:09:55 -07:00
fxmarty-amdGitHubClaude
 		2917d6363a [NVFP4][Hopper/AMD Instinct] Add Triton kernels for NVFP4 dequantization and QDQ emulation (#40033) 
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-04-30 17:35:48 -04:00
Chenxi QianGitHubmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		54146a9bf9 [Bugfix] correct h matrix layout in chunk_kda output kernel (#40956) 
Signed-off-by: ChenxiQian <chenxi.qian.cq@outlook.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-04-30 16:22:41 +08:00
Thien TranandGitHub 296741d025 [DSv4] Use cvt PTX for FP32->FP4 conversion (#41015) 
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
 2026-04-29 16:16:40 -07:00
Tianmu LiGitHubClaudeLi, Jiang <jiang1.li@intel.com>
 		22524f7a92 [Feat] CPU fp8 attn for AMX/AVX-512 (#39445) 
Signed-off-by: Li, Tianmu <tianmu.li@intel.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>
 2026-04-29 20:43:21 +08:00
ChaunceyandGitHub 92879e12ba [CI] fix test_rotary_embedding_opcheck format error (#41202) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2026-04-29 00:32:37 -07:00
Jee Jee LiandGitHub a269744e9f [Bugfix] Fix rope (#41113) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2026-04-28 22:42:35 -07:00
rasmithandGitHub 856b15c62c [CI][AMD][BugFix] Patch has_flashinfer decorator for test_select_rocm_aiter_backend (#41072) 
Signed-off-by: Randall Smith <Randall.Smith@amd.com>
 2026-04-29 02:12:17 +00:00
Matthew BonanniandGitHub fd74c90d9c [Attention][Spec Decode] Allow independent drafter attention backend selection (#39930) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-04-27 19:38:09 -07:00
Yongye ZhuandGitHub 706a04d34b [DSV4] Add silu clamp limit to shared expert (#40950) 
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
 2026-04-27 00:37:43 -07:00
Zhanda ZhuGitHubYubo Gao
 		5d5c776444 [Perf] FP8 FlashInfer Attn for ViT (#38065) 
Signed-off-by: Zhanda Zhu <zhandazhu@gmail.com>
Co-authored-by: Yubo Gao <ybgao-nvidia@users.noreply.github.com>
 2026-04-27 13:44:15 +08:00
+8
Yifan QiaoGitHubYongye ZhuYongye ZhuSimon MoBugen ZhaoGiancarlo DelfinJee Jee LiNick HillRoger WangRoy WangWoosuk KwonyoukaichaoZhewen LiZijing LiukhluuqizixiZhewen Li
 		4d51588e23 [Feat] DeepSeek V4 Rebased (#40860) 
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Signed-off-by: qizixi <zixi@inferact.ai>
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Yongye Zhu <yongye@inferact.ai>
Co-authored-by: Simon Mo <simon@inferact.ai>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Giancarlo Delfin <gdelfin@inferact.ai>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Roy Wang <yasong.wang@inferact.ai>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Zhewen Li <jerven.vllm@gmail.com>
Co-authored-by: Zijing Liu <liuzijing2014@gmail.com>
Co-authored-by: khluu <khluu000@gmail.com>
Co-authored-by: qizixi <zixi@inferact.ai>
Co-authored-by: Zhewen Li <zhewenli@inferact.ai>
 2026-04-26 18:31:08 -07:00
rasmithandGitHub 1e9f19ca3f [CI][AMD]BugFix] Fix deadlock occuring in test_moe_layer (#40767) 
Signed-off-by: Randall Smith <Randall.Smith@amd.com>
 2026-04-25 09:34:14 -04:00
Artem PerevedentsevandGitHub 333529deae [EPLB] Fix replica selection bias in fused_moe router (#40810) 
Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com>
 2026-04-24 22:06:41 +00:00
Jackmin801GitHubClaude
 		079a4cf399 [MoE] Move cutlass moe to fused_moe/experts/ (#40574) 
Signed-off-by: Jackmin801 <ongjackm@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-04-24 06:05:49 +00:00
Jackmin801GitHubRobert Shaw
 		447c372ac5 [MoE] Move remaining PrepareAndFinalize to prepare finalize folder (#39009) 
Signed-off-by: Robert Shaw <robertgshaw2@gmail.com>
Signed-off-by: Jackmin801 <ongjackm@gmail.com>
Co-authored-by: Robert Shaw <robertgshaw2@gmail.com>
 2026-04-23 20:00:53 -04:00
ℍ𝕠𝕝𝕝𝕠𝕨 𝕄𝕒𝕟GitHubmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Jee Jee Li
 		a250f1bd5f [Bugfix] LoRA for DeepSeek V3.2 (#35077) 
Signed-off-by: Hollow Man <hollowman@opensuse.org>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2026-04-22 19:33:50 +08:00
rasmithandGitHub 2463f00fb6 [AMD][CI][BugFix] Override normalize_e4m3fn_to_e4m3fnuz for fnuz machines in test_moe_layer_no_parallel (#40550) 
Signed-off-by: Randall Smith <Randall.Smith@amd.com>
 2026-04-22 02:21:02 +00:00
Jhao-Ting ChenandGitHub 46794958f0 test: add nan/inf clamp regression test for fused_topk_bias (#40553) 
Signed-off-by: Jhao-Ting Chen <jhaotingc@nvidia.com>
 2026-04-22 00:46:53 +00:00
bnellnmGitHubRobert Shaw
 		9db4650e5e [MoE Refactor] Add more MoE layer tests (#39349) 
Signed-off-by: Bill Nell <bnell@redhat.com>
Signed-off-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2026-04-21 18:12:36 -04:00
bnellnmandGitHub 5e584ce9ec [MoE Refactor] Remove SharedFusedMoE class (#35782) 
Signed-off-by: Bill Nell <bnell@redhat.com>
 2026-04-21 18:12:12 -04:00
Vadim GimpelsonandGitHub 6d85b36a9f Revert #38730 and #38791 (#40032) 
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>
Signed-off-by: Vadim Gimpelson <156319763+vadiklyutiy@users.noreply.github.com>
 2026-04-21 11:44:11 -04:00
Jhao-Ting ChenandGitHub 28c222157b fix: clamp NaN/Inf in topk_softmax to prevent duplicate expert IDs (#39391) 
Signed-off-by: Jhao-Ting Chen <jhaotingc@nvidia.com>
 2026-04-21 15:04:41 +04:00
Yanan CaoGitHubTheresa ShanClaude Opus 4.6
 		fe5c115ee4 [vLLM IR] Add IR op testing and benchmarking infrastructure (#40167) 
Signed-off-by: Yanan Cao <gmagogsfm@gmail.com>
Co-authored-by: Theresa Shan <Theresa.Shan@amd.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-21 00:23:03 +00:00
baiandGitHub 191e3fdaa1 Update flashinfer to 0.6.8 (#39959) 
Signed-off-by: bai <v@gor.io>
 2026-04-20 10:37:23 -07:00
bnellnmGitHubRobert Shaw
 		726efe177b [MoE Refactor] Move the shared/fused expert output sum into MoERunnerBase (#35949) 
Signed-off-by: Bill Nell <bnell@redhat.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2026-04-20 12:28:46 -04:00
Yan MaandGitHub e06de7f005 [XPU] enable triton attention test on XPU by removing cuda device binding (#39627) 
Signed-off-by: Yan Ma <yan.ma@intel.com>
 2026-04-20 20:57:11 +08:00
daniserebandGitHub d1135a5087 Fix MoE backend selection for LoRA (unquantized MoE) (#40273) 
Signed-off-by: Daniel Serebrenik <daserebrenik@nvidia.com>
 2026-04-19 17:18:40 +00:00
TJianandGitHub 45232a454e [FEAT] [Perf] [Gemma4] Fused Gemma4 Routing Function Triton (#39083) 
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
 2026-04-19 09:57:39 +00:00
Michael GoinandGitHub 48a65ccb02 [CI] Speed up test_fused_marlin_moe (#40178) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
 2026-04-17 19:26:21 -07:00
Michael GoinandGitHub 1f45e83756 Remove outdated tests test_mixtral_moe and test_duplicated_ignored_sequence_group (#40175) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-04-17 16:49:43 -07:00
Michael GoinandGitHub a8bffaa133 [Kernel] Add MXFP4 W4A4 CUTLASS MoE kernel for SM100 (#37463) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-04-17 16:42:32 -07:00
sychen52andGitHub 6b2b7bd0eb Add nvfp4 support to reshape_and_cache_flash (#37332) 
Signed-off-by: Shiyang Chen <shiychen@nvidia.com>
 2026-04-17 07:28:00 -07:00
bnellnmandGitHub 79e799ebbd [Bugfix] Temporarily disable B200 fp4 MoE layer tests (#40057) 
Signed-off-by: Bill Nell <bnell@redhat.com>
 2026-04-16 19:26:55 -04:00
roikoren755andGitHub b897f00c9c Gate SSU dispatch setup (#40039) 
Signed-off-by: Roi Koren <roik@nvidia.com>
 2026-04-16 13:06:01 -07:00
Yanan CaoGitHubClaude Opus 4.6
 		edc3648966 [Kernel][Helion] Fix inductor fusion of Helion HOP (#39944) 
Signed-off-by: Yanan Cao <gmagogsfm@gmail.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-16 04:41:26 -07:00
Fadi ArafehGitHubmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		445b7093fd [perf][cpu] Accelerate BF16 GELU with LUT impl on Arm CPUs (#37469) 
Signed-off-by: Fadi Arafeh <fadi.arafeh@arm.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-04-15 22:26:17 -07:00
bnellnmandGitHub 19ec9a0a62 [MoE Refactor] Refactor ZeroExpertFusedMoE into new framework (#35549) 
Signed-off-by: Bill Nell <bnell@redhat.com>
 2026-04-14 16:11:20 -04:00
Jackmin801GitHubRobert Shaw
 		1a9353bb02 [MoE] Move GPT OSS Triton kernel experts into fused_moe/experts/ (#39007) 
Signed-off-by: Robert Shaw <robertgshaw2@gmail.com>
Signed-off-by: Jackmin801 <ongjackm@gmail.com>
Co-authored-by: Robert Shaw <robertgshaw2@gmail.com>
 2026-04-14 19:27:39 +00:00
roikoren755andGitHub ecf5ff7ce3 [Mamba] Flashinfer selective_state_update (#36162) 
Signed-off-by: Roi Koren <roik@nvidia.com>
 2026-04-14 15:10:58 -04:00
Micah WilliamsonandGitHub 4f4f8eaa78 [ROCm][CI] Fix condition for test_per_token_group_quant_fp8_packed (#39730) 
Signed-off-by: Micah Williamson <micah.williamson@amd.com>
 2026-04-14 16:14:31 +00:00
bnellnmGitHubRobert Shaw
 		e1e318af01 [MoE Refactor] Remove MoE DP chunking (#39107) 
Signed-off-by: Bill Nell <bnell@redhat.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2026-04-14 09:48:05 -04:00
fxmarty-amdandGitHub 4eee77b877 [fix][MOE] Fix MOE experts intermediate_size dimension not being narrowed before weight loading (#39688) 
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
 2026-04-14 09:35:28 +00:00