khluu and Claude Opus 4.6
7c36acf7fd
Fix Python version regex quoting in Dockerfile
...
The grep -oP pattern with escaped double quotes was getting mangled
by Dockerfile shell escaping, causing the resolved version to include
a trailing quote character (e.g. "3.12.13\""), resulting in a 404
when downloading the source tarball. Switch to grep -oE matching
just the version number directly.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
Signed-off-by: khluu <khluu000@gmail.com >
2026-05-01 18:41:43 -07:00
khluu and Claude Opus 4.6
9f876610de
Build Python from source instead of using deadsnakes PPA
...
Replace the slow/flaky add-apt-repository deadsnakes PPA with building
Python from source. This avoids the long wait times and retry loops
when Launchpad PPA servers are slow or unresponsive.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
Signed-off-by: khluu <khluu000@gmail.com >
2026-05-01 18:31:44 -07:00
khluu
e7c9359432
p
...
Signed-off-by: khluu <khluu000@gmail.com >
2026-05-01 17:27:19 -07:00
khluu
747e379546
retry 10 times
...
Signed-off-by: khluu <khluu000@gmail.com >
2026-05-01 15:54:13 -07:00
Yongye Zhu and khluu
513b5f82dd
[DSV4] Add knob to enable pre-attn gemm ( #41443 )
...
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
(cherry picked from commit bcf5cac9fb
2026-05-01 12:24:17 -07:00
khluu
135722d58d
Revert "."
...
This reverts commit a5e0ea09ea
2026-05-01 12:23:33 -07:00
Rohan Potdar and khluu
7758daca7c
[ROCm][Bugfix][GPTOSS]: fix input_ids and expert_map args for quark w4a8 gptoss ( #41165 )
...
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com >
(cherry picked from commit 3795d7acf4
2026-05-01 12:17:45 -07:00
Isotr0py and khluu
5f79634b3b
[Perf] Intergrate Tile Kernels head_compute_mix_kernel for Deepseek-V4 ( #41255 )
...
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: Roger Wang <hey@rogerw.io >
(cherry picked from commit a9484dac7b
2026-05-01 12:17:28 -07:00
Bugen Zhao and khluu
05adde9955
[Bugfix] Pass reasoning parser kwargs to structured output ( #41199 )
...
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: wang.yuqi <yuqi.wang@daocloud.io >
(cherry picked from commit a07642667d
2026-05-01 12:14:44 -07:00
Jiangyun Zhu and khluu
9cdcda5fc2
[Bugfix] BailingMoeV2.5: rotate full qk_rope_head_dim in MLA RoPE ( #41185 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
(cherry picked from commit 6d7d4da99e
2026-05-01 12:13:50 -07:00
JiangWeixiang and khluu
a5d4a264dd
[Bugfix ] fix bailing_moe_linear ( #40859 )
...
Signed-off-by: ghphotoframe <854746559@qq.com >
(cherry picked from commit ed57f77192
2026-05-01 12:13:41 -07:00
Yongye Zhu and khluu
a5e0ea09ea
.
...
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
(cherry picked from commit db0c9d52d4khluu000@gmail.com >
2026-04-30 23:16:02 -07:00
Yongye Zhu and khluu
28a1a85d89
[DSV4] Add BF16 and MXFP8 A2A support for flashinfer a2a one sided ( #40960 )
...
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
Signed-off-by: Zijing Liu <liuzijing2014@gmail.com >
Co-authored-by: Zijing Liu <liuzijing2014@users.noreply.github.com >
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com >
(cherry picked from commit b4806c8ee1
2026-04-30 21:39:14 -07:00
Yongye Zhu and khluu
82ccd9f3b2
[Bugfix] Fix persistent_topk cooperative deadlock at TopK=1024 ( #41189 )
...
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com >
(cherry picked from commit a749a33d8d
2026-04-30 21:39:14 -07:00
Thien Tran and khluu
d9f3961abb
[DSv4] Use cvt PTX for FP32->FP4 conversion ( #41015 )
...
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg >
(cherry picked from commit 296741d025
2026-04-30 21:39:14 -07:00
Jee Jee Li and khluu
4fee1390f4
[Bugfix] Fix repeated DSv4 RoPE cache initialization ( #41148 )
...
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com >
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
(cherry picked from commit 9d8ad5b408
2026-04-30 21:39:14 -07:00
Chauncey and khluu
bf590b6a24
[Bugfix] DSV32/V4 add missing type conversion for non-streaming tool calls ( #41198 )
...
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com >
(cherry picked from commit 762022cafb
2026-04-30 21:39:13 -07:00
Wei Zhao and khluu
66df2c03f5
[Bugfix] Fix max_num_batched_token not captured in cuda graph ( #40734 )
...
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com >
Signed-off-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com >
Co-authored-by: Wei Zhao (Engrg-Hardware 1) <weizha@login-bia02.bia.clusters.nvidia.com >
(cherry picked from commit 8b49cf3a37
2026-04-30 21:39:13 -07:00
Jiangyun Zhu and khluu
17896d817b
[Bugfix] fix inductor error for dpsk v4 ( #41135 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
(cherry picked from commit 2ae73c758c
2026-04-30 21:39:13 -07:00
Wei Zhao and khluu
1a131a066c
[Bugfix] Fix Deepseek V4 import error due to AOT compile cache loading ( #41090 )
...
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com >
Signed-off-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com >
(cherry picked from commit 803b9d7881
2026-04-30 21:39:13 -07:00
qizixi and khluu
6fffb0b8c8
[DSV4] Align aux stream API with DeepseekV4DecoderLayer ( #41171 )
...
Signed-off-by: zixi-qi <zixi@inferact.ai >
(cherry picked from commit 6fb3f7b46b
2026-04-30 21:39:13 -07:00
Nick Hill and khluu
a076bd1e67
[Core] Account for num_gpu_blocks_override in max_model_len checks ( #41069 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
(cherry picked from commit e68fa1b90a
2026-04-30 21:39:13 -07:00
Yongye Zhu and khluu
c244b6fc83
[DSV4] Enable Multi-stream for Pre-Attn GEMM ( #41061 )
...
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
(cherry picked from commit 5aa371dc8e
2026-04-30 21:39:13 -07:00
Jee Jee Li and khluu
15cc87d2fe
[Model][DSV4] Support base model ( #41006 )
...
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com >
(cherry picked from commit 2c8b76c5cb
2026-04-30 21:39:13 -07:00
youkaichao and khluu
c68fa39ced
Auto-disable expandable_segments around cumem memory pool ( #40812 )
...
Signed-off-by: youkaichao <youkaichao@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
(cherry picked from commit 2ce95a761b
2026-04-30 21:39:13 -07:00
Yongye Zhu and GitHub
05ebca5250
Temporary disable persistent topk ( #41442 )
...
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
2026-04-30 20:37:19 -07:00
Zhewen Li and khluu
88d34c6409
[Docker] Install numactl CLI in CUDA runtime image ( #41032 )
...
Signed-off-by: Zhewen Li <zhewenli@inferact.ai >
Co-authored-by: Zhewen Li <zhewenli@inferact.ai >
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com >
(cherry picked from commit 6232fb4b66
2026-04-27 12:04:32 -07:00
Yongye Zhu and khluu
b8160878f0
[DSV4] Add silu clamp limit to shared expert ( #40950 )
...
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
(cherry picked from commit 706a04d34b
2026-04-27 00:40:46 -07:00
Dao007forever and khluu
84c276d7ea
[Bugfix] Cap SWA/chunked-local runtime admission to startup pool-sizing bound ( #40946 )
...
Signed-off-by: Dao Le <Dao007forever@gmail.com >
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
(cherry picked from commit 7b1bc0a3eb
2026-04-26 21:43:00 -07:00
khluu
5eb3657578
Revert "[Frontend] Remove frontend pooling multi task support. ( #37861 )"
...
This reverts commit d2e2e856ad
2026-04-26 21:42:15 -07:00
+8
4d51588e23
[Feat] DeepSeek V4 Rebased ( #40860 )
...
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
Signed-off-by: qizixi <zixi@inferact.ai >
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com >
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Yongye Zhu <yongye@inferact.ai >
Co-authored-by: Simon Mo <simon@inferact.ai >
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Giancarlo Delfin <gdelfin@inferact.ai >
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Roger Wang <hey@rogerw.io >
Co-authored-by: Roy Wang <yasong.wang@inferact.ai >
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai >
Co-authored-by: youkaichao <youkaichao@gmail.com >
Co-authored-by: Zhewen Li <jerven.vllm@gmail.com >
Co-authored-by: Zijing Liu <liuzijing2014@gmail.com >
Co-authored-by: khluu <khluu000@gmail.com >
Co-authored-by: qizixi <zixi@inferact.ai >
Co-authored-by: Zhewen Li <zhewenli@inferact.ai >
2026-04-26 18:31:08 -07:00
32e45636e3
[torch.compile]: Disable Sequence Parallelism (SP) for piecewise compilation ( #38373 )
...
Signed-off-by: SouthWest7 <am1ao@qq.com >
Signed-off-by: Xinan Miao <1403572259@qq.com >
Co-authored-by: SouthWest7 <am1ao@qq.com >
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Wang Xingran <72983099+wangxingran222@users.noreply.github.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-04-26 17:44:42 +00:00
b39c266dae
[KV Offload] Offload all KV blocks when doing prefill in P/D ( #40346 )
...
Signed-off-by: omerpaz95 <omerpaz95@gmail.com >
Signed-off-by: omerpaz95 <73347585+omerpaz95@users.noreply.github.com >
Co-authored-by: Or Ozeri <or@ozery.com >
2026-04-26 15:06:01 +03:00
Dao007forever and GitHub
9558f43903
[Bugfix] Size FlashInfer NVLink MNNVL workspace to EP group ( #40893 )
...
Signed-off-by: Dao Le <Dao007forever@gmail.com >
2026-04-26 01:26:34 -07:00
Jee Jee Li and GitHub
8cd174fa35
[LoRA] MoE LoRA Refactor ( #40338 )
2026-04-26 01:55:19 +00:00
c798593f0d
[Bugfix] Fix the DSML token leakage in DSV4/3.2 ( #40806 )
...
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com >
Signed-off-by: sfeng33 <4florafeng@gmail.com >
Co-authored-by: sfeng33 <4florafeng@gmail.com >
Co-authored-by: Windswithyou 1694599440@qq.com
2026-04-26 08:58:50 +08:00
12a3f6454b
[Bugfix][MoE] Only unpad routed output before shared expert add or routed output transform ( #40865 )
...
Signed-off-by: Netanel Haber <58652339+netanel-haber@users.noreply.github.com >
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com >
2026-04-25 20:50:12 +00:00
Or Ozeri and GitHub
60cd878a3b
[kv_offload+HMA][11/N]: Support store with multiple KV groups ( #39403 )
...
Signed-off-by: Or Ozeri <oro@il.ibm.com >
2026-04-25 20:00:46 +03:00
rasmith and GitHub
1e9f19ca3f
[CI][AMD]BugFix] Fix deadlock occuring in test_moe_layer ( #40767 )
...
Signed-off-by: Randall Smith <Randall.Smith@amd.com >
2026-04-25 09:34:14 -04:00
labAxiaoming and GitHub
6646c0c7e0
[Opt] Optimize deepstack buffer handling for multimodal Qwen3 models ( #40145 )
...
Signed-off-by: xiaoming <1259730330@qq.com >
2026-04-25 21:04:26 +08:00
Andreas Karatzas and GitHub
95995bbef8
[ROCm][Engine] Fix GPU memory leaks in engine shutdown and test workaround for async KV prefix cache reset ( #38503 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
2026-04-25 05:25:20 +00:00
07351e0883
[Feature] Warm up readonly multimodal processor during renderer startup ( #40797 )
...
Signed-off-by: Chenguang ZHENG <645327136@qq.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-04-25 03:57:41 +00:00
Andreas Karatzas and GitHub
428b988c98
[ROCm][CI] Fix trust_remote_code AttributeError in EAGLE3 acceptance length test ( #40306 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
2026-04-25 02:59:31 +00:00
Andreas Karatzas and GitHub
e54894fc85
[ROCm][CI] Fix TestSiluMulGroupFp8QuantModel after W8A8 block linear refactor ( #39799 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
2026-04-25 11:20:59 +09:00
Angela Yi and GitHub
bc2ae5a3d6
[Test] Increase qwen2_vl num_logprobs to fix torch 2.12 update ( #40818 )
...
Signed-off-by: Angela Yi <angelayi@meta.com >
2026-04-25 00:59:20 +00:00
Wentao Ye and GitHub
a474da2813
[Refactor] Remove unused dead code ( #40640 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-04-25 07:28:18 +08:00
Lucas Kabela and GitHub
ce6a199ecc
[BE][Bugfix] Respect TORCH_COMPILE_DISABLE env var at the vLLM config level for torch 2.12 ( #40715 )
...
Signed-off-by: Lucas Kabela <lucaskabela@meta.com >
2026-04-24 16:25:03 -07:00
Ignacio Sica and GitHub
f88763efc3
[Bugfix] add seq_lens_cpu_upper_bound to CommonAttentionMetadata in mla_runner.py ( #40844 )
...
Signed-off-by: ignaciosica <mignacio.sica@gmail.com >
2026-04-24 23:13:52 +00:00
Artem Perevedentsev and GitHub
333529deae
[EPLB] Fix replica selection bias in fused_moe router ( #40810 )
...
Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com >
2026-04-24 22:06:41 +00:00
Zhang Jian and GitHub
8825608205
[Bugfix][CI] Fix wrong residual shape in TestFusedAddRMSNorm.example_inputs that causes flaky test ( #40629 )
...
Signed-off-by: Zhang Jian <jianmusings@gmail.com >
2026-04-24 16:40:07 -04:00
