 Woosuk Kwon
|
f6fa9700e6
|
Merge branch 'main' into woosuk/triton-fix
|
2026-06-17 23:38:48 +00:00 |
|
|
Woosuk Kwon
|
0221ab433e
|
wip
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
|
2026-06-17 22:57:43 +00:00 |
|
|
|
0d339cf135
|
[Bugfix] Fix NixlConnector handshake block_len validation for GQA-replicated KV heads (#45879)
Signed-off-by: Oseltamivir <58582368+Oseltamivir@users.noreply.github.com>
Co-authored-by: waynehacking8 <waynehacking8@gmail.com>
|
2026-06-17 15:11:29 -07:00 |
|
|
|
5fd21eb0b2
|
[BUG] fix hidden states nan for hybrid attention models (#45849)
Signed-off-by: shanjiaz <hezhao@redhat.com>
Co-authored-by: shanjiaz <hezhao@redhat.com>
|
2026-06-17 18:02:24 -04:00 |
|
|
Ting SUNandGitHub
|
9d4b87f4f0
|
[Bugfix][Model] Validate DefaultModelLoader / LoadConfig and fail with clear errors (#45196)
Signed-off-by: Ting Sun <suntcrick@gmail.com>
|
2026-06-17 21:46:33 +00:00 |
|
|
|
58b2e89642
|
[Bugfix][Gemma4] Render reasoning on assistant turns without tool_calls (#45867)
Signed-off-by: Luciano Martins <lucianommartins@users.noreply.github.com>
Co-authored-by: Luciano Martins <lucianommartins@users.noreply.github.com>
|
2026-06-17 20:44:15 +00:00 |
|
|
Wentao YeandGitHub
|
2659f60a1a
|
[Refactor] Remove dead quantization code and tests (#45454)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
|
2026-06-17 16:12:01 -04:00 |
|
|
|
091386a99b
|
[Bugfix] MiniMax-M3 (AMD): add packed_modules_mapping and pass swiglu… (#45794)
Signed-off-by: wangjiaxin99 <jiaxwang@amd.com>
Co-authored-by: TJian <tunjian.tan@embeddedllm.com>
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com>
|
2026-06-17 19:15:46 +00:00 |
|
|
qli88andGitHub
|
d112eb1ac7
|
[feature] MiniMax-M3-MXFP4 support added (#45896)
Signed-off-by: Qiang Li <qiang.li2@amd.com>
|
2026-06-17 18:50:48 +00:00 |
|
|
Wentao YeandGitHub
|
2a47a9ff0f
|
[DSV4 Perf] Optimize dsv4 cudagraph by reducing eager_break_during_capture, 26.8% ~ 27.9% E2E TTFT improvement (#45309)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
|
2026-06-17 09:34:53 -07:00 |
|
|
Wentao YeandGitHub
|
9c7c74bf10
|
[Log] Update deepgemm log (#45857)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
|
2026-06-17 15:34:22 +00:00 |
|
|
daniserebandGitHub
|
5e27b2baf4
|
[Bugfix] Pass TP group to FlashInfer all-reduce fusion (#45917)
Signed-off-by: Daniel Serebrenik <daserebrenik@nvidia.com>
|
2026-06-17 15:24:26 +00:00 |
|
|
zhanqiuhuandGitHub
|
eb0fdeb1e8
|
[Bugfix][PD] Fix DSV4 disaggregated serving (#45831)
Signed-off-by: ZhanqiuHu <zhu@redhat.com>
|
2026-06-17 15:17:14 +00:00 |
|
|
|
46f74e144b
|
[Kernel][Helion][1/N] Add Helion kernel for rms_norm_dynamic_per_token_quant (#34432)
Signed-off-by: Sean Chen <seachen@redhat.com>
Co-authored-by: Yanan Cao <gmagogsfm@gmail.com>
|
2026-06-17 23:03:54 +08:00 |
|
|
Wentao YeandGitHub
|
0a7bacdcac
|
[DSv4 Perf] DSv4 flashinfer sparse index cache for metadata, 2%~4% TTFT improvement (#45863)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
|
2026-06-17 10:55:48 -04:00 |
|
|
amirkl94andGitHub
|
8b2b566ea7
|
Feature: Enable Flashinfer non-gated MoE bf16 (#43853)
Signed-off-by: Amir Klein <203507526+amirkl94@users.noreply.github.com>
|
2026-06-17 14:32:49 +00:00 |
|
|
xaguilar-amdandGitHub
|
0b131b16c9
|
[ROCm][AITER][Quark] Tag per-channel FP8 weights as PER_CHANNEL so AITER pre-shuffled GEMM is selected (#44626)
Signed-off-by: Xavier Aguilar <xavier.aguilarfruto@amd.com>
|
2026-06-17 14:05:34 +00:00 |
|
|
|
bcb518ad7a
|
[quant][autoround]Refactor INC quantization into package with INCScheme orchestrator (#40601)
Signed-off-by: yiliu30 <yi4.liu@intel.com>
Signed-off-by: Zhenzhong1 <zhenzhong.xu@intel.com>
Signed-off-by: Zhenzhong Xu <zhenzhong.xu@intel.com>
Co-authored-by: n1ck-guo <heng.guo@intel.com>
Co-authored-by: Zhenzhong1 <zhenzhong.xu@intel.com>
|
2026-06-17 21:51:32 +08:00 |
|
|
Chaojun ZhangandGitHub
|
06e1e0885c
|
[XPU] Fix test_logprobs_e2e import error: pin lm-eval[api]>=0.4.12 (#44469)
Signed-off-by: Chaojun Zhang <chaojun.zhang@intel.com>
|
2026-06-17 12:26:47 +00:00 |
|
|
Isotr0pyandGitHub
|
1a59078c87
|
[CI/Build] Avoid duplicate ViT CG test introduced by accident (#45654)
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
|
2026-06-17 12:23:44 +00:00 |
|
|
Oğuzhan KIRandGitHub
|
fa85ead2f3
|
[MM][Perf][CG] Support ViT full CUDA graph for Kimi-VL (#41992)
Signed-off-by: oguz <oguzhankir17@gmail.com>
|
2026-06-17 12:14:01 +00:00 |
|
|
|
e28e8c8782
|
[ROCm][Quant] Minimax-M3: Enable fp8_per_channel for bf16 weights on mi300x (#45854)
Signed-off-by: Hongxia Yang <hongxia.yang@amd.com>
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: tjtanaa <tunjian.tan@embeddedllm.com>
|
2026-06-17 12:02:40 +00:00 |
|
|
Angelo RuoccoandGitHub
|
ee0fd6984a
|
docs, kv_offloading: add docs for selective offload (#45279)
Signed-off-by: Angelo Ruocco <ang@zurich.ibm.com>
|
2026-06-17 14:58:00 +03:00 |
|
|
vllmellmandGitHub
|
d537122398
|
[ROCm][Bugfix]: Fallback GFX942 sparse MLA ops to Triton (#45782)
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
|
2026-06-17 11:41:29 +00:00 |
|
|
Juan Pérez de AlgabaandGitHub
|
3d20275bb4
|
fix(security): enforce audio decode duration limit in chat completions path (#45908)
Signed-off-by: jperezde <jperezde@redhat.com>
|
2026-06-17 11:07:13 +00:00 |
|
|
|
f694d43b33
|
[Bugfix][test] Use Salesforce/wikitext for ppl tests (#45913)
Co-authored-by: wentian-byte <192079369+wentian-byte@users.noreply.github.com>
|
2026-06-17 10:37:19 +00:00 |
|
|
Nikhilesh ChhetriandGitHub
|
3c6084bb0d
|
[Bugfix][Gemma4] Pre-initialise streaming reasoning state when prompt ends inside an open <|channel> (fixes #45834) (#45852)
Signed-off-by: nikhilesh-csa <nchhetri@csa1.com>
|
2026-06-17 06:16:02 -04:00 |
|
|
Joel SmithandGitHub
|
68ff30d40e
|
[Bugfix] Fixes MiniCPM-O resampler device placement to avoid tensor device mismatch (#42332)
Signed-off-by: j9smith <j.smith9103@outlook.com>
|
2026-06-17 08:35:27 +00:00 |
|
|
|
6d8fff5698
|
[KV Connector][Offloading] Avoid blocking the engine to flush offloads on idle (#45595)
Signed-off-by: Itay Etelis <itay.etelis@ibm.com>
Signed-off-by: Or Ozeri <oro@il.ibm.com>
Signed-off-by: Itay Etelis <Itay.etelis@gmail.com>
Co-authored-by: Itay Etelis <itay.etelis@ibm.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
Co-authored-by: Itay Etelis <Itay.etelis@gmail.com>
|
2026-06-17 11:35:07 +03:00 |
|
|
|
e2c58570ea
|
[Rust Frontend] Support hybrid/external DP LB in Python supervised bootstrap (#45805)
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-06-17 07:32:40 +00:00 |
|
|
Taneem IbrahimandGitHub
|
43fa24e832
|
[Misc] Validate Cohere Embed Mixed Content Payloads (#45873)
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
|
2026-06-17 06:57:12 +00:00 |
|
|
arghyadeep sarkarandGitHub
|
93bbe94d3a
|
[Kernel] Add weightless RMSNorm CUDA kernels for has_weight=False (#41430) (#44109)
Signed-off-by: hello-args <args.sarkar@gmail.com>
|
2026-06-16 23:45:55 -07:00 |
|
|
Will EatonandGitHub
|
17bc144556
|
[Rust Frontend] Add serde defaults for omit_defaults fields in EngineCoreSamplingParams (#45848)
Signed-off-by: Will Eaton <weaton@redhat.com>
|
2026-06-17 06:40:49 +00:00 |
|
|
Sahil SinghandGitHub
|
295232a26a
|
[Rust Frontend] Add /abort_requests endpoint (#44382)
Signed-off-by: Sahil Singh <sahiilsiingh37@gmail.com>
|
2026-06-17 06:40:47 +00:00 |
|
|
ReidandGitHub
|
56e4345226
|
[Rust Frontend] Support prompt-only completions (#44938)
Signed-off-by: reidliu41 <reid201711@gmail.com>
|
2026-06-17 06:38:06 +00:00 |
|
|
Nick HillandGitHub
|
e9993a52aa
|
[BugFix][CI] Fix scheduler plugin test (#45897)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
|
2026-06-17 06:30:49 +00:00 |
|
|
|
a46abb7ae6
|
[Bugfix][Quantization] Reject unsupported compressed tensors KV cache schemes (#45312)
Signed-off-by: Ting Sun <suntcrick@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-06-17 05:08:41 +00:00 |
|
|
|
4c62663315
|
[M3] Enable FP8 sparse GQA (#45744)
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
|
2026-06-16 21:38:03 -07:00 |
|
|
|
d78650cf97
|
[CI][NIXL] Pin NIXL to 1.2.0 (#45843)
Signed-off-by: Itay Alroy <ialroy@nvidia.com>
Signed-off-by: Itay Alroy <75032521+itayalroy@users.noreply.github.com>
Co-authored-by: ovidiusm <ovidium@nvidia.com>
|
2026-06-16 21:29:34 -07:00 |
|
|
|
5bdc01bcc3
|
[M3] Tune Triton indexer score decode for spec-decode (#45743)
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-06-16 21:07:34 -07:00 |
|
|
liangel-02andGitHub
|
20a5f8b43b
|
[FlexAttention] make custom mask mods fully cudagraphable (#45232)
Signed-off-by: Angel Li <liangel@meta.com>
|
2026-06-17 11:53:12 +08:00 |
|
|
|
7b5d60cc37
|
[Bugfix][V1] Clean up compiled-model bytecode hooks on VllmRunner exit (#45195)
Signed-off-by: Ting Sun <suntcrick@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-06-16 20:31:17 -07:00 |
|
|
Nick HillandGitHub
|
14b438a98b
|
[ModelRunnerV2] Various model/config compatibility fixes (#45868)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
|
2026-06-17 03:23:01 +00:00 |
|
|
|
2785a5e0e6
|
[Bugfix][ROCm] Fix FP8 per-tensor scale rank mismatch causing Inductor assertion failure (#44912)
Signed-off-by: nehmathe2 <nehmathe2@gmail.com>
Signed-off-by: Divakar Verma <divakar.verma@amd.com>
Signed-off-by: nehmathe <nehmathe@amd.com>
Co-authored-by: TJian <tunjian.tan@embeddedllm.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Divakar Verma <divakar.verma@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
|
2026-06-16 20:17:42 -07:00 |
|
|
|
efd15e192a
|
[Bugfix][ROCm] Fix MiniMax-M3 FP8 KV cache dtype (#45720)
Signed-off-by: Cam Quilici <cjquilici@gmail.com>
Signed-off-by: Cameron Quilici <cjquilici@gmail.com>
Co-authored-by: Hongxia Yang <62075498+hongxiayang@users.noreply.github.com>
Co-authored-by: TJian <tunjian.tan@embeddedllm.com>
|
2026-06-17 03:14:45 +00:00 |
|
|
|
556b063e45
|
[XPU] Fix test_spec_decode_logprobs: use FLASH_ATTN for XPU in GPU_DETERMINISM_KWARGS (#44468)
Signed-off-by: Chaojun Zhang <chaojun.zhang@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
|
2026-06-17 11:07:04 +08:00 |
|
|
|
aa0ac8a661
|
[CI] Run pre-commit on self-hosted vllm-runners (#45865)
Signed-off-by: khluu <khluu000@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
2026-06-16 19:49:22 -07:00 |
|
|
FedericoandGitHub
|
b831374cf1
|
[Bugfix][Gemma4] Fix parsing when thinking is disabled (#45832)
Signed-off-by: Federico Iezzi <fiezzi@google.com>
|
2026-06-17 02:41:36 +00:00 |
|
|
|
71bc19dbdd
|
[Bugfix] Fix MoE model load OOM in FlashInfer_TRTLLM backend with sleep mode (#45589)
Signed-off-by: Dakai An <dakaian108@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-06-16 19:36:51 -07:00 |
|
|
Kunshang JiandGitHub
|
ef2c40dc00
|
[XPU][CI] fix server test file path (#45870)
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
|
2026-06-17 09:06:25 +08:00 |
|