karylab_agents/vllm
1
0
Fork 0
forked from Karylab-cklius/vllm
Code Pull Requests 2 Actions 1 Packages Activity
16,131 Commits 354 Branches 175 Tags
khluu/release-v0.20.1-uv-python
Commit Graph
16131 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
khluuandClaude Opus 4.7 797cdc15f0 [CI] Use uv-installed standalone Python in vllm-base (drop deadsnakes) 
The vllm-base stage installs python3.12 via the deadsnakes PPA, which
requires reaching launchpad.net (REST API) or ppa.launchpadcontent.net
(CDN). On the release-v2 build agents, both are unreachable
(Connection timed out to 185.125.190.80), causing the regular Ubuntu
22.04 release images to hang or fail at the apt-get update step.

Switch vllm-base to the same approach the `base` stage already uses:
uv installs a standalone python3.12 (via python-build-standalone,
hosted on astral.sh CDN) into /opt/venv, and we symlink it to
/usr/bin/python3. No Launchpad dependency.

Side benefits:
- Consistent python install path between `base` and `vllm-base`.
- Removes the software-properties-common / add-apt-repository step.
- Drops unused DEADSNAKES_MIRROR_URL / DEADSNAKES_GPGKEY_URL /
  GET_PIP_URL stage args.

Test branch: khluu/release-v0.20.1-uv-python (off releases/v0.20.1).

Signed-off-by: khluu <khluu000@gmail.com>

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-01 13:42:32 -07:00
khluuandClaude Opus 4.7 438f6548c1 [CI] Use deadsnakes mirror for non-Ubuntu24.04 release images 
The 4 Ubuntu 22.04 release image builds (CUDA 12.9 / 13.0 x both archs)
hang for ~30min and intermittently fail because
`add-apt-repository ppa:deadsnakes/ppa` hits the launchpad.net REST API
to resolve PPA metadata, and that call has been timing out in CI:

    TimeoutError: [Errno 110] Connection timed out
    ...
    E: Unable to locate package python3.12

Plumb DEADSNAKES_MIRROR_URL and DEADSNAKES_GPGKEY_URL through to those
steps so the Dockerfile takes the mirror branch and writes the apt
source directly, skipping the flaky API call. The mirror points at the
public Launchpad CDN (same packages add-apt-repository would have
configured) and the GPG key comes from keyserver.ubuntu.com.

Ubuntu 24.04 builds are unaffected — they ship python3.12 in default
apt repos and never enter the deadsnakes branch.

Signed-off-by: khluu <khluu000@gmail.com>

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-01 13:25:05 -07:00
Yongye Zhuandkhluu 513b5f82dd [DSV4] Add knob to enable pre-attn gemm (#41443) 
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
(cherry picked from commit bcf5cac9fb)
 2026-05-01 12:24:17 -07:00
khluu 135722d58d Revert "." 
This reverts commit a5e0ea09ea.
 2026-05-01 12:23:33 -07:00
Rohan Potdarandkhluu 7758daca7c [ROCm][Bugfix][GPTOSS]: fix input_ids and expert_map args for quark w4a8 gptoss (#41165) 
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com>
(cherry picked from commit 3795d7acf4)
 2026-05-01 12:17:45 -07:00
Isotr0pyandkhluu 5f79634b3b [Perf] Intergrate Tile Kernels head_compute_mix_kernel for Deepseek-V4 (#41255) 
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
(cherry picked from commit a9484dac7b)
 2026-05-01 12:17:28 -07:00
Bugen Zhaoandkhluu 05adde9955 [Bugfix] Pass reasoning parser kwargs to structured output (#41199) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: wang.yuqi <yuqi.wang@daocloud.io>
(cherry picked from commit a07642667d)
 2026-05-01 12:14:44 -07:00
Jiangyun Zhuandkhluu 9cdcda5fc2 [Bugfix] BailingMoeV2.5: rotate full qk_rope_head_dim in MLA RoPE (#41185) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
(cherry picked from commit 6d7d4da99e)
 2026-05-01 12:13:50 -07:00
JiangWeixiangandkhluu a5d4a264dd [Bugfix ] fix bailing_moe_linear (#40859) 
Signed-off-by: ghphotoframe <854746559@qq.com>
(cherry picked from commit ed57f77192)
 2026-05-01 12:13:41 -07:00
Yongye Zhuandkhluu a5e0ea09ea . 
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>

(cherry picked from commit db0c9d52d4)

Signed-off-by: khluu <khluu000@gmail.com>
 2026-04-30 23:16:02 -07:00
Yongye Zhuandkhluu 28a1a85d89 [DSV4] Add BF16 and MXFP8 A2A support for flashinfer a2a one sided (#40960) 
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Signed-off-by: Zijing Liu <liuzijing2014@gmail.com>
Co-authored-by: Zijing Liu <liuzijing2014@users.noreply.github.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
(cherry picked from commit b4806c8ee1)
 2026-04-30 21:39:14 -07:00
Yongye Zhuandkhluu 82ccd9f3b2 [Bugfix] Fix persistent_topk cooperative deadlock at TopK=1024 (#41189) 
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
(cherry picked from commit a749a33d8d)
 2026-04-30 21:39:14 -07:00
Thien Tranandkhluu d9f3961abb [DSv4] Use cvt PTX for FP32->FP4 conversion (#41015) 
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
(cherry picked from commit 296741d025)
 2026-04-30 21:39:14 -07:00
Jee Jee Liandkhluu 4fee1390f4 [Bugfix] Fix repeated DSv4 RoPE cache initialization (#41148) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
(cherry picked from commit 9d8ad5b408)
 2026-04-30 21:39:14 -07:00
Chaunceyandkhluu bf590b6a24 [Bugfix] DSV32/V4 add missing type conversion for non-streaming tool calls (#41198) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
(cherry picked from commit 762022cafb)
 2026-04-30 21:39:13 -07:00
Wei Zhaoandkhluu 66df2c03f5 [Bugfix] Fix max_num_batched_token not captured in cuda graph (#40734) 
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Signed-off-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com>
Co-authored-by: Wei Zhao (Engrg-Hardware 1) <weizha@login-bia02.bia.clusters.nvidia.com>
(cherry picked from commit 8b49cf3a37)
 2026-04-30 21:39:13 -07:00
Jiangyun Zhuandkhluu 17896d817b [Bugfix] fix inductor error for dpsk v4 (#41135) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
(cherry picked from commit 2ae73c758c)
 2026-04-30 21:39:13 -07:00
Wei Zhaoandkhluu 1a131a066c [Bugfix] Fix Deepseek V4 import error due to AOT compile cache loading (#41090) 
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Signed-off-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com>
(cherry picked from commit 803b9d7881)
 2026-04-30 21:39:13 -07:00
qizixiandkhluu 6fffb0b8c8 [DSV4] Align aux stream API with DeepseekV4DecoderLayer (#41171) 
Signed-off-by: zixi-qi <zixi@inferact.ai>
(cherry picked from commit 6fb3f7b46b)
 2026-04-30 21:39:13 -07:00
Nick Hillandkhluu a076bd1e67 [Core] Account for num_gpu_blocks_override in max_model_len checks (#41069) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
(cherry picked from commit e68fa1b90a)
 2026-04-30 21:39:13 -07:00
Yongye Zhuandkhluu c244b6fc83 [DSV4] Enable Multi-stream for Pre-Attn GEMM (#41061) 
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
(cherry picked from commit 5aa371dc8e)
 2026-04-30 21:39:13 -07:00
Jee Jee Liandkhluu 15cc87d2fe [Model][DSV4] Support base model (#41006) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
(cherry picked from commit 2c8b76c5cb)
 2026-04-30 21:39:13 -07:00
youkaichaoandkhluu c68fa39ced Auto-disable expandable_segments around cumem memory pool (#40812) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
(cherry picked from commit 2ce95a761b)
 2026-04-30 21:39:13 -07:00
Yongye ZhuandGitHub 05ebca5250 Temporary disable persistent topk (#41442) 
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
 2026-04-30 20:37:19 -07:00
Zhewen Liandkhluu 88d34c6409 [Docker] Install numactl CLI in CUDA runtime image (#41032) 
Signed-off-by: Zhewen Li <zhewenli@inferact.ai>
Co-authored-by: Zhewen Li <zhewenli@inferact.ai>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
(cherry picked from commit 6232fb4b66)
v0.20.0 		2026-04-27 12:04:32 -07:00
Yongye Zhuandkhluu b8160878f0 [DSV4] Add silu clamp limit to shared expert (#40950) 
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
(cherry picked from commit 706a04d34b)
 2026-04-27 00:40:46 -07:00
Dao007foreverandkhluu 84c276d7ea [Bugfix] Cap SWA/chunked-local runtime admission to startup pool-sizing bound (#40946) 
Signed-off-by: Dao Le <Dao007forever@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
(cherry picked from commit 7b1bc0a3eb)
 2026-04-26 21:43:00 -07:00
khluu 5eb3657578 Revert "[Frontend] Remove frontend pooling multi task support. (#37861)" 
This reverts commit d2e2e856ad.
 2026-04-26 21:42:15 -07:00
+8
Yifan QiaoGitHubYongye ZhuYongye ZhuSimon MoBugen ZhaoGiancarlo DelfinJee Jee LiNick HillRoger WangRoy WangWoosuk KwonyoukaichaoZhewen LiZijing LiukhluuqizixiZhewen Li
 		4d51588e23 [Feat] DeepSeek V4 Rebased (#40860) 
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Signed-off-by: qizixi <zixi@inferact.ai>
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Yongye Zhu <yongye@inferact.ai>
Co-authored-by: Simon Mo <simon@inferact.ai>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Giancarlo Delfin <gdelfin@inferact.ai>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Roy Wang <yasong.wang@inferact.ai>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Zhewen Li <jerven.vllm@gmail.com>
Co-authored-by: Zijing Liu <liuzijing2014@gmail.com>
Co-authored-by: khluu <khluu000@gmail.com>
Co-authored-by: qizixi <zixi@inferact.ai>
Co-authored-by: Zhewen Li <zhewenli@inferact.ai>
 2026-04-26 18:31:08 -07:00
Xinan MiaoGitHubSouthWest7gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>OpenAI CodexWang Xingranmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		32e45636e3 [torch.compile]: Disable Sequence Parallelism (SP) for piecewise compilation (#38373) 
Signed-off-by: SouthWest7 <am1ao@qq.com>
Signed-off-by: Xinan Miao <1403572259@qq.com>
Co-authored-by: SouthWest7 <am1ao@qq.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Wang Xingran <72983099+wangxingran222@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-04-26 17:44:42 +00:00
omerpaz95GitHubOr Ozeri
 		b39c266dae [KV Offload] Offload all KV blocks when doing prefill in P/D (#40346) 
Signed-off-by: omerpaz95 <omerpaz95@gmail.com>
Signed-off-by: omerpaz95 <73347585+omerpaz95@users.noreply.github.com>
Co-authored-by: Or Ozeri <or@ozery.com>
 2026-04-26 15:06:01 +03:00
Dao007foreverandGitHub 9558f43903 [Bugfix] Size FlashInfer NVLink MNNVL workspace to EP group (#40893) 
Signed-off-by: Dao Le <Dao007forever@gmail.com>
 2026-04-26 01:26:34 -07:00
Jee Jee LiandGitHub 8cd174fa35 [LoRA] MoE LoRA Refactor (#40338) 2026-04-26 01:55:19 +00:00
ChaunceyGitHubsfeng33Windswithyou 1694599440@qq.com
 		c798593f0d [Bugfix] Fix the DSML token leakage in DSV4/3.2 (#40806) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
Signed-off-by: sfeng33 <4florafeng@gmail.com>
Co-authored-by: sfeng33 <4florafeng@gmail.com>
Co-authored-by: Windswithyou 1694599440@qq.com
 2026-04-26 08:58:50 +08:00
Netanel HaberGitHubJee Jee Li
 		12a3f6454b [Bugfix][MoE] Only unpad routed output before shared expert add or routed output transform (#40865) 
Signed-off-by: Netanel Haber <58652339+netanel-haber@users.noreply.github.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2026-04-25 20:50:12 +00:00
Or OzeriandGitHub 60cd878a3b [kv_offload+HMA][11/N]: Support store with multiple KV groups (#39403) 
Signed-off-by: Or Ozeri <oro@il.ibm.com>
 2026-04-25 20:00:46 +03:00
rasmithandGitHub 1e9f19ca3f [CI][AMD]BugFix] Fix deadlock occuring in test_moe_layer (#40767) 
Signed-off-by: Randall Smith <Randall.Smith@amd.com>
 2026-04-25 09:34:14 -04:00
labAxiaomingandGitHub 6646c0c7e0 [Opt] Optimize deepstack buffer handling for multimodal Qwen3 models (#40145) 
Signed-off-by: xiaoming <1259730330@qq.com>
 2026-04-25 21:04:26 +08:00
Andreas KaratzasandGitHub 95995bbef8 [ROCm][Engine] Fix GPU memory leaks in engine shutdown and test workaround for async KV prefix cache reset (#38503) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-04-25 05:25:20 +00:00
Chenguang ZhengGitHubOpenAI Codex
 		07351e0883 [Feature] Warm up readonly multimodal processor during renderer startup (#40797) 
Signed-off-by: Chenguang ZHENG <645327136@qq.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-04-25 03:57:41 +00:00
Andreas KaratzasandGitHub 428b988c98 [ROCm][CI] Fix trust_remote_code AttributeError in EAGLE3 acceptance length test (#40306) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-04-25 02:59:31 +00:00
Andreas KaratzasandGitHub e54894fc85 [ROCm][CI] Fix TestSiluMulGroupFp8QuantModel after W8A8 block linear refactor (#39799) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-04-25 11:20:59 +09:00
Angela YiandGitHub bc2ae5a3d6 [Test] Increase qwen2_vl num_logprobs to fix torch 2.12 update (#40818) 
Signed-off-by: Angela Yi <angelayi@meta.com>
 2026-04-25 00:59:20 +00:00
Wentao YeandGitHub a474da2813 [Refactor] Remove unused dead code (#40640) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-04-25 07:28:18 +08:00
Lucas KabelaandGitHub ce6a199ecc [BE][Bugfix] Respect TORCH_COMPILE_DISABLE env var at the vLLM config level for torch 2.12 (#40715) 
Signed-off-by: Lucas Kabela <lucaskabela@meta.com>
 2026-04-24 16:25:03 -07:00
Ignacio SicaandGitHub f88763efc3 [Bugfix] add seq_lens_cpu_upper_bound to CommonAttentionMetadata in mla_runner.py (#40844) 
Signed-off-by: ignaciosica <mignacio.sica@gmail.com>
 2026-04-24 23:13:52 +00:00
Artem PerevedentsevandGitHub 333529deae [EPLB] Fix replica selection bias in fused_moe router (#40810) 
Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com>
 2026-04-24 22:06:41 +00:00
Zhang JianandGitHub 8825608205 [Bugfix][CI] Fix wrong residual shape in TestFusedAddRMSNorm.example_inputs that causes flaky test (#40629) 
Signed-off-by: Zhang Jian <jianmusings@gmail.com>
 2026-04-24 16:40:07 -04:00
qli88andGitHub 095d2f87e8 [Bug] Fix GLM-5.1 running error on ROCm platform (#40763) 
Signed-off-by: Qiang Li <qiang.li2@amd.com>
 2026-04-24 19:54:40 +00:00
Neil SchemenauerGitHubSimon Mo
 		21792520e7 [Build] Add Python 3.14 to supported version list. (#34770) 
Signed-off-by: Neil Schemenauer <nas@arctrix.com>
Co-authored-by: Simon Mo <simon.mo@hey.com>
 2026-04-24 10:24:05 -07:00