karylab_agents/vllm
1
0
Fork 0
forked from Karylab-cklius/vllm
Code Pull Requests 2 Actions 1 Packages Activity
18,945 Commits 354 Branches 175 Tags
cuda-arch-fixup
Commit Graph
46 Commits
Author SHA1 Message Date
Shengqi ChenandGitHub a02155c787 Merge branch 'main' into cuda-arch-fixup 2026-07-21 09:33:48 +08:00
flutist-alibabaGitHubCopilot Autofix powered by AIShengqi Chen
 		0d9210a502 Fixes non-coalesced HBM access in marlin_int4_fp8_preprocess_kernel_awq (#47268) 
Signed-off-by: xjx <493337577@qq.com>
Signed-off-by: flutist-alibaba <30485581+flutist@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Shengqi Chen <harry-chen@outlook.com>
 2026-07-20 18:30:38 -07:00
Shengqi ChenandGitHub 401bed48ad Merge branch 'main' into cuda-arch-fixup 
Signed-off-by: Shengqi Chen <harry-chen@outlook.com>
 2026-07-03 15:54:56 +08:00
Chris LeonardandGitHub fbc9ba6d30 New stable abi cleanup (#46656) 
Signed-off-by: Chris Leonard <chleonar@redhat.com>
 2026-07-03 14:02:26 +08:00
Shengqi Chen a10e369f06 [Build] Address CUDA arch review comments 
Fix CUDA arch warning tests so they do not depend on importing the stable libtorch extension, which keeps the warning coverage active in lightweight CI environments and avoids mypy treating a fixture value as a base class.

Use regex for the compiled-arch parser, apply formatter output, and make the CUTLASS grouped GEMM Python support query fall back to false when the op is unavailable or unimplemented in the current build.

Signed-off-by: Shengqi Chen <harry-chen@outlook.com>
 2026-06-30 20:03:28 +08:00
Shengqi ChenandCodex 0f471a3088 [Build] Scope stable CUDA kernel feature macros 
Keep optional stable CUDA kernel feature macros on the source files that consume them instead of adding them to VLLM_GPU_FLAGS. This avoids perturbing unrelated compile commands and invalidating more cache entries when optional kernel families change.

Also align CUTLASS grouped MoE support with the SM10x/SM11x family so Thor works under both CUDA 12 SM101 and CUDA 13 SM110 reporting, and remove the stale ENABLE_CUTLASS_MLA definition left after the old CUTLASS MLA path was deleted.

Co-authored-by: Codex <codex@openai.com>

Signed-off-by: Shengqi Chen <harry-chen@outlook.com>
 2026-06-30 19:27:42 +08:00
Jee Jee LiandGitHub 23aed9b0ee [Kernel] Enable PDL for per_token_group_quant_8bit_kernel (#46508) 
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
 2026-06-25 08:42:51 +08:00
Cyrus LeungandGitHub 24d5186138 [Bugfix] Re-enable FP8 MoE on NVIDIA Thor (#46339) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-06-24 07:35:46 -07:00
JasonLi314andGitHub 93bad11912 [Bugfix] Fix gridDim.y overflow for large row counts (#45255) 
Signed-off-by: Jason Li <li.jason.cs@gmail.com>
 2026-06-19 23:27:45 -04:00
Chris LeonardandGitHub b9a7cd464c [12/n] final _C library kernel migration (#45415) 2026-06-19 06:57:26 -07:00
HumphreyandGitHub 4583630b56 [Bugfix][Kernel] Check output alignment in vectorize_with_alignment (fixes misaligned-address crash for non-multiple-of-8 head sizes) (#45466) 
Signed-off-by: HumphreySun98 <humphreysun98@gmail.com>
 2026-06-18 16:58:22 +00:00
Jee Jee LiandGitHub 22cc891108 [Kernel] Add PDL support for DeepGEMM kernel (#46006) 
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
 2026-06-18 20:49:01 +08:00
Micah WilliamsonandGitHub e945169207 Revert "[Kernel] Add PDL support for DeepGEMM kernel" (#45999) 2026-06-17 22:59:48 -07:00
Jee Jee LiandGitHub 4403af8fb5 [Kernel] Add PDL support for DeepGEMM kernel (#42996) 
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
 2026-06-17 20:37:17 -07:00
Xin HeGitHubmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Kunshang Ji
 		5ed15f42b9 Fix the E8M0 scale computation in the MXFP4 (W4A4) MOE CUTLASS kernel (#43557) 
Signed-off-by: Xin He <xin3.he@intel.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-06-15 06:04:54 -07:00
Shengqi ChenGitHubXin LiShawRongChange72
 		78e7293bb1 [Build] Fix CUDA arch build coverage gaps (#45277) 
Signed-off-by: Shengqi Chen <harry-chen@outlook.com>
Co-authored-by: Xin Li <xinli-sw@users.noreply.github.com>
Co-authored-by: ShawRong <ShawRong@users.noreply.github.com>
Co-authored-by: Change72 <Change72@users.noreply.github.com>
 2026-06-13 22:09:20 -07:00
Wentao YeGitHubmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		71b961dd35 [Perf] SM90 cutlass fp8 mm supports odd M by swap_ab, 180~290% kernel performance improvement (#44572) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-06-13 12:05:45 -07:00
Isotr0pyandGitHub 6635279d8a [Migration] Migrate GGUF quantization support to plugin (#39612) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-06-12 12:02:21 -07:00
Chris LeonardGitHubShengqi Chen
 		a014dddbaa [11b/n] Migrate Machete kernels to torch stable ABI (#45304) 
Signed-off-by: Chris Leonard <chleonar@redhat.com>
Signed-off-by: Shengqi Chen <harry-chen@outlook.com>
Co-authored-by: Shengqi Chen <harry-chen@outlook.com>
 2026-06-12 10:36:49 +00:00
Chris LeonardandGitHub 7021be66e8 [11a/n] Migrate Marlin kernels to torch stable ABI (#45176) 
Signed-off-by: Chris Leonard <chleonar@redhat.com>
 2026-06-11 21:22:37 -07:00
Juan Pérez de AlgabaandGitHub f219788f91 [Security] Fix info disclosure via int32 truncation in GGUF dequantize kernels (#44971) 
Signed-off-by: jperezde <jperezde@redhat.com>
 2026-06-11 08:05:14 +00:00
Micah WilliamsonandGitHub e1ed89dbee Revert "[Kernel] Speed up silu_and_mul_per_block_quant with warp-shuf… (#45066) 
Signed-off-by: Micah Williamson <micah.williamson@amd.com>
 2026-06-09 14:12:06 -07:00
SII-yangdianGitHubSII-yangdian
 		a4b14b98c6 [Kernel] Speed up silu_and_mul_per_block_quant with warp-shuffle reduction + vectorized I/O (#44173) 
Signed-off-by: SII-yangdian <yangdian@sii.edu.cn>
Co-authored-by: SII-yangdian <yangdian@sii.edu.cn>
 2026-06-09 10:41:26 -07:00
Chris LeonardandGitHub 56aff0dd15 [10/n] Migrate cuda_view and silu_and_mul_per_block_quant kernels to torch stale ABI. (#44334) 2026-06-04 20:14:43 -07:00
Charlie FuandGitHub 71df063c49 Enable perf_token_group_quant/_C_stable_libtorch for ROCm (#42758) 
Signed-off-by: charlifu <charlifu@amd.com>
 2026-06-02 23:23:28 -07:00
Chris LeonardandGitHub 4d93bc35c9 Migrate header files to torch stable abi (#44013) 2026-06-02 08:09:52 -07:00
Chris LeonardGitHubMikayla GawareckiShengqi Chen
 		22a58640b4 [9/n] Migrate attention and cache kernels to torch stable ABI (continued) (#43717) 
Signed-off-by: Chris Leonard <chleonar@redhat.com>
Signed-off-by: Mikayla Gawarecki <mikaylagawarecki@gmail.com>
Co-authored-by: Mikayla Gawarecki <mikaylagawarecki@gmail.com>
Co-authored-by: Shengqi Chen <harry-chen@outlook.com>
 2026-05-29 04:44:45 +00:00
Jakub ZakrzewskiGitHubWentao YeYongye Zhu
 		5bb8d2767a [Kernel] Batch invariant NVFP4 linear using cutlass (#39912) 
Signed-off-by: Jakub Zakrzewski <jzakrzewski@nvidia.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
 2026-05-23 09:41:12 -04:00
Chris LeonardGitHubMikayla GawareckiShengqi Chen
 		a7be0f342d [7/n] Migrate pos_encoding and norm kernels to libtorch stable ABI (continued) (#43209) 
Signed-off-by: Mikayla Gawarecki <mikaylagawarecki@gmail.com>
Signed-off-by: Chris Leonard <chleonar@redhat.com>
Co-authored-by: Mikayla Gawarecki <mikaylagawarecki@gmail.com>
Co-authored-by: Shengqi Chen <harry-chen@outlook.com>
 2026-05-23 13:20:00 +08:00
Chris LeonardGitHubMikayla GawareckiShengqi Chen
 		07aeaf9d4d [6/n] Migrate activation kernels, gptq, gguf, non cutlass w8a8 to libtorch stable ABI (continued) (#42663) 
Signed-off-by: Mikayla Gawarecki <mikaylagawarecki@gmail.com>
Signed-off-by: Chris Leonard <chleonar@redhat.com>
Co-authored-by: Mikayla Gawarecki <mikaylagawarecki@gmail.com>
Co-authored-by: Shengqi Chen <harry-chen@outlook.com>
 2026-05-20 00:18:12 -07:00
Wentao YeandGitHub 37ece593c1 [Perf] Padded nvfp4 quant kernel to remove additional copy, 2.4%~5.7% e2e performance improvement (#42774) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-05-18 16:38:12 -07:00
Blake LeddenGitHubClaude Opus 4.6Pavani Majety
 		06d020bb6e [Bugfix] Fix SM121 (DGX Spark) exclusion from Marlin/CUTLASS FP8 paths (#35568) 
Signed-off-by: Blake Ledden <blake@secondnaturecomputing.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Pavani Majety <pmajety@nvidia.com>
 2026-05-15 10:59:00 -07:00
Chris LeonardGitHubMikayla Gawarecki
 		85b2fecab7 [5/n] Migrate CUTLASS MLA, hadamard, awq, allspark and DSV3 fused a gemm to torch stable ABI (continued) (#42339) 
Signed-off-by: Mikayla Gawarecki <mikaylagawarecki@gmail.com>
Co-authored-by: Mikayla Gawarecki <mikaylagawarecki@gmail.com>
 2026-05-13 07:24:39 +00:00
Jiahan Chang (Cyrus)GitHubYongye Zhu
 		dd6b3a5ef5 [Perf] Use 2D-grid to eliminate divmod in W8W8 group quant (#42153) 
Signed-off-by: jiahanc <173873397+jiahanc@users.noreply.github.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
 2026-05-12 10:01:30 -04:00
Wentao YeandGitHub 0d453e2336 [Perf] Batch invariance with Cutlass fp8 support, 28.9% E2E latency improvement (#40408) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2026-05-11 12:20:58 -04:00
Yongye ZhuGitHubRoger Wang
 		a3c83ff2fd Faster per-token fp8 group quant packed kernel for blackwell (#41326) 
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
 2026-04-30 18:09:55 -07:00
Cyrus LeungandGitHub 4a42aba380 [CI/Build] Enable FP8 on NVIDIA Thor (#39712) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-04-29 09:48:52 -07:00
Jakub ZakrzewskiandGitHub 6fbec8ed47 [Bugfix][Kernel] nvfp4 cutlass MoE: fix nvfp4 experts quant out-of-bounds read for expert counts not divisible by 4 or 16 (#40351) 
Signed-off-by: Jakub Zakrzewski <jzakrzewski@nvidia.com>
 2026-04-21 19:06:09 +00:00
mysterious hhhhGitHubmgoinClaude Opus 4.7
 		d0359f3e04 [Bugfix] Guard mxfp4_experts_quant bindings on ENABLE_NVFP4_SM100 (#40191) 
Signed-off-by: ultranationalism <www913363043@gmail.com>
Signed-off-by: mgoin <mike.goin12@gmail.com>
Co-authored-by: mgoin <mike.goin12@gmail.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-18 13:58:46 -07:00
Michael GoinandGitHub a8bffaa133 [Kernel] Add MXFP4 W4A4 CUTLASS MoE kernel for SM100 (#37463) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-04-17 16:42:32 -07:00
Wei ZhaoandGitHub 59b2f7b640 [Perf] Fuse Zero Initializer for FP8 DeepGemm Block Quant Kernel (#39547) 
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
 2026-04-11 07:16:51 -07:00
NecofishandGitHub 580090db6b [Kernel] Add swapAB support for SM120 CUTLASS blockwise FP8 GEMM (#38325) 2026-04-03 15:49:59 +02:00
Wentao YeandGitHub c9a9db0e02 [Compile] Fix nvfp4 compile warning (#38573) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-04-01 18:28:57 +00:00
mikaylagawareckiandGitHub 7c080dd3c5 [4/n] Migrate FP4/W4A8 CUTLASS kernels to torch stable ABI (#37503) 
Signed-off-by: Mikayla Gawarecki <mikaylagawarecki@gmail.com>
 2026-03-31 10:21:13 -07:00
mikaylagawareckiandGitHub ab1a6a43fa [3/n] Migrate cutlass/scaled_mm_entry.cu torch stable ABI (#37221) 
Signed-off-by: Mikayla Gawarecki <mikaylagawarecki@gmail.com>
 2026-03-30 11:20:13 -07:00
mikaylagawareckiandGitHub bf4cc9ed2d [2/n] Migrate per_token_group_quant to torch stable ABI (#36058) 
Signed-off-by: Mikayla Gawarecki <mikaylagawarecki@gmail.com>
 2026-03-25 10:15:13 -07:00