|
a3c83ff2fd
|
Faster per-token fp8 group quant packed kernel for blackwell (#41326)
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
|
2026-04-30 18:09:55 -07:00 |
|
|
Woosuk KwonandGitHub
|
9c61864bf8
|
[DeepSeek] Use torch.mm for bf16xbf16->fp32 gemm (#41300)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
|
2026-04-30 16:28:57 -07:00 |
|
|
|
a749a33d8d
|
[Bugfix] Fix persistent_topk cooperative deadlock at TopK=1024 (#41189)
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
2026-04-29 21:03:45 -07:00 |
|
|
Cyrus LeungandGitHub
|
4a42aba380
|
[CI/Build] Enable FP8 on NVIDIA Thor (#39712)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2026-04-29 09:48:52 -07:00 |
|
|
|
22524f7a92
|
[Feat] CPU fp8 attn for AMX/AVX-512 (#39445)
Signed-off-by: Li, Tianmu <tianmu.li@intel.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>
|
2026-04-29 20:43:21 +08:00 |
|
|
Jee Jee LiandGitHub
|
a269744e9f
|
[Bugfix] Fix rope (#41113)
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
|
2026-04-28 22:42:35 -07:00 |
|
|
Yongye ZhuandGitHub
|
706a04d34b
|
[DSV4] Add silu clamp limit to shared expert (#40950)
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
|
2026-04-27 00:37:43 -07:00 |
|
|
+8
|
4d51588e23
|
[Feat] DeepSeek V4 Rebased (#40860)
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Signed-off-by: qizixi <zixi@inferact.ai>
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Yongye Zhu <yongye@inferact.ai>
Co-authored-by: Simon Mo <simon@inferact.ai>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Giancarlo Delfin <gdelfin@inferact.ai>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Roy Wang <yasong.wang@inferact.ai>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Zhewen Li <jerven.vllm@gmail.com>
Co-authored-by: Zijing Liu <liuzijing2014@gmail.com>
Co-authored-by: khluu <khluu000@gmail.com>
Co-authored-by: qizixi <zixi@inferact.ai>
Co-authored-by: Zhewen Li <zhewenli@inferact.ai>
|
2026-04-26 18:31:08 -07:00 |
|
|
Woosuk KwonandGitHub
|
7d3195ea9f
|
[Bugfix] Fix IMA in DSA + MTP (#40772)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
|
2026-04-24 01:40:20 -07:00 |
|
|
Xin YangandGitHub
|
cf8a613a87
|
Support only half types for concat_mla_q kernel (#37892)
Signed-off-by: Xin Yang <xyangx@amazon.com>
|
2026-04-23 23:51:05 -07:00 |
|
|
|
2f314bc5e6
|
[CPU] Added faster exp routine for lower precision data types. (#38112)
Signed-off-by: Anna Mayne <anna.mayne@arm.com>
Co-authored-by: Fadi Arafeh <fadi.arafeh@arm.com>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>
|
2026-04-23 13:14:44 +00:00 |
|
|
Matthias GehreandGitHub
|
4b7869d6bc
|
[ROCm] Add gfx1102/gfx1103 support (#40037)
Signed-off-by: Matthias Gehre <matthias.gehre@amd.com>
|
2026-04-23 01:32:04 -07:00 |
|
|
lyd1992andGitHub
|
04eac6ba24
|
[Bugfix][CPU][RISC-V] Clamp exp() input to prevent NaN (#40428)
Signed-off-by: liuyudong <liuyudong@iscas.ac.cn>
|
2026-04-22 09:38:18 +00:00 |
|
|
Jakub ZakrzewskiandGitHub
|
6fbec8ed47
|
[Bugfix][Kernel] nvfp4 cutlass MoE: fix nvfp4 experts quant out-of-bounds read for expert counts not divisible by 4 or 16 (#40351)
Signed-off-by: Jakub Zakrzewski <jzakrzewski@nvidia.com>
|
2026-04-21 19:06:09 +00:00 |
|
|
Jhao-Ting ChenandGitHub
|
28c222157b
|
fix: clamp NaN/Inf in topk_softmax to prevent duplicate expert IDs (#39391)
Signed-off-by: Jhao-Ting Chen <jhaotingc@nvidia.com>
|
2026-04-21 15:04:41 +04:00 |
|
|
velonica0andGitHub
|
ec7aafc02a
|
[CPU][RISC-V] Support multiple RVV VLEN targets via compile-time dispatch (#39478)
Signed-off-by: velonica0 <like@mail.nankai.edu.cn>
|
2026-04-20 14:36:59 +08:00 |
|
|
|
d0359f3e04
|
[Bugfix] Guard mxfp4_experts_quant bindings on ENABLE_NVFP4_SM100 (#40191)
Signed-off-by: ultranationalism <www913363043@gmail.com>
Signed-off-by: mgoin <mike.goin12@gmail.com>
Co-authored-by: mgoin <mike.goin12@gmail.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
2026-04-18 13:58:46 -07:00 |
|
|
Michael GoinandGitHub
|
a8bffaa133
|
[Kernel] Add MXFP4 W4A4 CUTLASS MoE kernel for SM100 (#37463)
Signed-off-by: mgoin <mgoin64@gmail.com>
|
2026-04-17 16:42:32 -07:00 |
|
|
sychen52andGitHub
|
6b2b7bd0eb
|
Add nvfp4 support to reshape_and_cache_flash (#37332)
Signed-off-by: Shiyang Chen <shiychen@nvidia.com>
|
2026-04-17 07:28:00 -07:00 |
|
|
Li, JiangandGitHub
|
d02421a7db
|
[CPU] Refactor CPU affinity and memory management (#39781)
Signed-off-by: jiang1.li <jiang1.li@intel.com>
|
2026-04-17 21:01:08 +08:00 |
|
|
R3hankhanandGitHub
|
4b7ca37bd4
|
[CPU][IBM Z][Dockefile][Docs] Fix s390x builds for torch 2.11 and update docs for s390x (#39910)
Signed-off-by: Rehan Khan <Rehan.Khan7@ibm.com>
|
2026-04-15 22:26:21 -07:00 |
|
|
|
445b7093fd
|
[perf][cpu] Accelerate BF16 GELU with LUT impl on Arm CPUs (#37469)
Signed-off-by: Fadi Arafeh <fadi.arafeh@arm.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-04-15 22:26:17 -07:00 |
|
|
|
4beeb0689c
|
fused qknorm+rope kernel optimization for SM9.0 (#37376)
Signed-off-by: EricccYang <yangyang4991@gmail.com>
Signed-off-by: Kaicheng Yang <53411596+EricccYang@users.noreply.github.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
|
2026-04-12 19:58:37 -07:00 |
|
|
|
bd8bd52308
|
[Bugfix] Runtime driver check for cuMemcpyBatchAsync in swap_blocks_batch (#38919)
Signed-off-by: Itay Etelis <itay.etelis@ibm.com>
Co-authored-by: Itay Etelis <itay.etelis@ibm.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
|
2026-04-11 11:02:34 -06:00 |
|
|
Wei ZhaoandGitHub
|
59b2f7b640
|
[Perf] Fuse Zero Initializer for FP8 DeepGemm Block Quant Kernel (#39547)
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
|
2026-04-11 07:16:51 -07:00 |
|
|
Jee Jee LiandGitHub
|
ecd1ea1363
|
[Kernel] Porting the TRTLLM minimax_allreduce_rms kernels (#37045)
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
|
2026-04-11 00:20:20 +08:00 |
|
|
Isotr0pyandGitHub
|
58c0a928c9
|
[Bugfix] Fix broken explicit unquantized kv cache dtype support (#38922)
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
|
2026-04-09 22:27:53 -07:00 |
|
|
|
f83de7196f
|
[BugFix] Fix OOB read in CUTLASS grouped GEMM with epilogue (#38571)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
|
2026-04-09 23:52:52 -04:00 |
|
|
Ganesh RandGitHub
|
445a2a4d1a
|
feat(cpu): add CPU support for draft model speculative decoding (#32662)
Signed-off-by: R <Ganesh.R@amd.com>
|
2026-04-10 11:49:52 +08:00 |
|
|
Yongye ZhuandGitHub
|
2e98406048
|
[Refactor] Improve indexer decode path metadata preparation (#38865)
|
2026-04-08 20:49:15 -07:00 |
|
|
|
b55d830ec7
|
[Perf][Kernel] Persistent TopK scheduler: unified CUDAGraph-safe kernel with dynamic per-row dispatch - DeepSeek-V3.2 DSA decode (#37421)
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com>
Signed-off-by: Roberto L. Castro <38211239+LopezCastroRoberto@users.noreply.github.com>
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
|
2026-04-08 13:35:57 -04:00 |
|
|
Gregory ShtrasbergandGitHub
|
56c976c1b5
|
[ROCm] Enable fused_silu_mul_block_quant on ROCm (#38817)
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
|
2026-04-08 11:23:32 -05:00 |
|
|
Andrey TalmanandGitHub
|
2111997f96
|
[release 2.11] Update to torch 2.11 (#34644)
|
2026-04-07 18:55:48 -07:00 |
|
|
Kyle MylonakisandGitHub
|
7b9de7c892
|
[Bugfix] Correct mistake in chained comparison in static assert logic (#38699)
Signed-off-by: Kyle Mylonakis <kyle@protopia.ai>
|
2026-04-07 18:24:39 +08:00 |
|
|
Wentao YeandGitHub
|
062f1a2d70
|
[Bug] Fix compile error for swap_blocks_batch in CUDA 13 (#38915)
|
2026-04-03 16:56:38 -07:00 |
|
|
NecofishandGitHub
|
580090db6b
|
[Kernel] Add swapAB support for SM120 CUTLASS blockwise FP8 GEMM (#38325)
|
2026-04-03 15:49:59 +02:00 |
|
|
Anton IvanovandGitHub
|
abebd9323d
|
[CPU] Replace OMP initialization (#36487)
Signed-off-by: Anton Ivanov <anton.ivanov@cambridgegreys.com>
|
2026-04-03 18:42:43 +08:00 |
|
|
|
4a06e1246e
|
[Perf] Batch KV cache swap copies via cuMemcpyBatchAsync (#38460)
Signed-off-by: Itay Etelis <itay.etelis@ibm.com>
Co-authored-by: Itay Etelis <itay.etelis@ibm.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
|
2026-04-03 03:13:23 +00:00 |
|
|
Carl YandGitHub
|
3bc2734dd0
|
[Kernel] Fuse FP8 output quantization into merge_attn_states (#36518)
Signed-off-by: Carl You <4531192+carlyou@users.noreply.github.com>
|
2026-04-03 01:47:04 +00:00 |
|
|
Li, JiangandGitHub
|
c6f722b93e
|
[CPU] Support gelu act in cpu_fused_moe (#38770)
Signed-off-by: jiang1.li <jiang1.li@intel.com>
|
2026-04-02 14:14:32 +08:00 |
|
|
Xin YangandGitHub
|
9bd7231106
|
Revert "[Kernel] Add gpt-oss Router GEMM kernel (#37205)" (#38778)
Signed-off-by: Xin Yang <xyangx@amazon.com>
|
2026-04-01 22:02:32 -07:00 |
|
|
|
3aab680e3e
|
[ROCm][Bugfix] Fix ROCm runtime failure due to missing symbol (#38750)
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
Signed-off-by: Gregory Shtrasberg <156009573+gshtras@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: tjtanaavllm <tunjian.tan@amd.com>
|
2026-04-01 21:30:11 -07:00 |
|
|
MonishverandGitHub
|
c09ad767cd
|
Feature/silu block quant fusion v1 (#32996)
Signed-off-by: Monishver Chandrasekaran <monishverchandrasekaran@gmail.com>
|
2026-04-01 18:50:43 +00:00 |
|
|
Wentao YeandGitHub
|
c9a9db0e02
|
[Compile] Fix nvfp4 compile warning (#38573)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
|
2026-04-01 18:28:57 +00:00 |
|
|
Michael GoinandGitHub
|
db5d0719e1
|
[Kernel] Add MXFP8 to Marlin GEMM/MoE and refactor Mxfp8LinearOp (#34664)
Signed-off-by: mgoin <mgoin64@gmail.com>
|
2026-04-01 09:41:42 -07:00 |
|
|
Li, JiangandGitHub
|
36d7f19897
|
[CPU] Support head_size 512 in cpu_attn (#38676)
Signed-off-by: jiang1.li <jiang1.li@intel.com>
|
2026-04-01 05:42:27 +00:00 |
|
|
Olya KozlovaandGitHub
|
598190aac3
|
[fix] Remove trtllm ragged mla prefills (#36540)
Signed-off-by: Olya Kozlova <okozlova@nvidia.com>
|
2026-03-31 12:30:27 -07:00 |
|
|
mikaylagawareckiandGitHub
|
7c080dd3c5
|
[4/n] Migrate FP4/W4A8 CUTLASS kernels to torch stable ABI (#37503)
Signed-off-by: Mikayla Gawarecki <mikaylagawarecki@gmail.com>
|
2026-03-31 10:21:13 -07:00 |
|
|
Yintong LuandGitHub
|
f09daea261
|
[CPU] Support int8 compute mode in CPU AWQ (#35697)
Signed-off-by: Yintong Lu <yintong.lu@intel.com>
|
2026-03-31 15:27:37 +08:00 |
|
|
|
bcc6f67447
|
[Bugfix] Use null block (0) for padded block table entries (#35431)
Signed-off-by: SandishKumarHN <sandish@fb.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
|
2026-03-30 14:02:51 -07:00 |
|