karylab_agents/vllm
1
0
Fork 0
forked from Karylab-cklius/vllm
Code Pull Requests 2 Actions 1 Packages Activity
16,133 Commits 354 Branches 175 Tags
releases/v0.20.1-python-from-source
Commit Graph
130 Commits
Author SHA1 Message Date
khluuandClaude Opus 4.6 7c36acf7fd Fix Python version regex quoting in Dockerfile 
The grep -oP pattern with escaped double quotes was getting mangled
by Dockerfile shell escaping, causing the resolved version to include
a trailing quote character (e.g. "3.12.13\""), resulting in a 404
when downloading the source tarball. Switch to grep -oE matching
just the version number directly.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Signed-off-by: khluu <khluu000@gmail.com>
 2026-05-01 18:41:43 -07:00
khluuandClaude Opus 4.6 9f876610de Build Python from source instead of using deadsnakes PPA 
Replace the slow/flaky add-apt-repository deadsnakes PPA with building
Python from source. This avoids the long wait times and retry loops
when Launchpad PPA servers are slow or unresponsive.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Signed-off-by: khluu <khluu000@gmail.com>
 2026-05-01 18:31:44 -07:00
khluu 747e379546 retry 10 times 
Signed-off-by: khluu <khluu000@gmail.com>
 2026-05-01 15:54:13 -07:00
Zhewen Liandkhluu 88d34c6409 [Docker] Install numactl CLI in CUDA runtime image (#41032) 
Signed-off-by: Zhewen Li <zhewenli@inferact.ai>
Co-authored-by: Zhewen Li <zhewenli@inferact.ai>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
(cherry picked from commit 6232fb4b66)
 2026-04-27 12:04:32 -07:00
Dmitry TokarevandGitHub 6dec49f27e [Build] Bump CUDA to 13.0.2 to match PyTorch 2.11.0 (#40669) 
Signed-off-by: Dmitry Tokarev <dtokarev@nvidia.com>
 2026-04-24 10:27:11 +00:00
JohnnyandGitHub 7f95a66cbf [NVIDIA] Add sm_110 (Jetson Thor) to CUDA 13.0 build targets (#39233) 2026-04-23 15:42:14 -04:00
Shengqi ChenGitHubClaude Opus 4.6
 		3ed5231c6a [Build] Switch default CUDA to 13.0, update CUDA architecture lists, clean up stale build-args (#39878) 
Signed-off-by: Shengqi Chen <harry-chen@outlook.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-23 15:51:28 +08:00
baiandGitHub 191e3fdaa1 Update flashinfer to 0.6.8 (#39959) 
Signed-off-by: bai <v@gor.io>
 2026-04-20 10:37:23 -07:00
Harry MellorGitHubkhluuCyrus Leungjiang1.li
 		03f8d3a548 Update to transformers v5 (#30566) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: khluu <khluu000@gmail.com>
Signed-off-by: Kevin H. Luu <khluu000@gmail.com>
Signed-off-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: khluu <khluu000@gmail.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: jiang1.li <jiang1.li@intel.com>
 2026-04-15 16:29:15 -07:00
Harry MellorGitHubRobert Shaw
 		fc645f1acc Add structure to requirements/ directory (#39024) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2026-04-10 13:46:41 -07:00
Zhewen LiandGitHub 9e78555743 [Docker] Add fastsafetensors to NVIDIA Dockerfile (#38950) 2026-04-08 22:21:37 -07:00
Michael GoinGitHubClaude
 		eb4205fee5 [UX] Integrate DeepGEMM into vLLM wheel via CMake (#37980) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-04-08 18:56:32 -07:00
Andrey TalmanandGitHub 2111997f96 [release 2.11] Update to torch 2.11 (#34644) 2026-04-07 18:55:48 -07:00
JohnnyandGitHub b4a2f3ac36 [NVIDIA] Bugfix NVFP4 DGX Spark and RTX50 (#38423) 
Signed-off-by: johnnynunez <johnnynuca14@gmail.com>
Signed-off-by: Johnny <johnnynuca14@gmail.com>
 2026-03-30 09:36:18 -07:00
Michael GoinandGitHub 2bf5b70ae8 [CI Bugfix] Pre-download missing FlashInfer headers in Docker build (#38391) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-03-28 06:09:00 -07:00
Michael GoinandGitHub 0f0e03890e [UX] Add flashinfer-cubin as CUDA default dep (#37233) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-03-24 14:13:08 -07:00
amey asgaonkarandGitHub 0c1809c806 Add Ubuntu 24.04 support for Docker builds (#35386) 
Signed-off-by: aasgaonkar <aasgaonkar@nvidia.com>
 2026-03-24 13:34:44 -07:00
HariandGitHub a3e2e250f0 [Feature] Add Azure Blob Storage support for RunAI Model Streamer (#34614) 
Signed-off-by: hasethuraman <hsethuraman@microsoft.com>
 2026-03-15 19:38:21 +08:00
Dimitrios BariamisGitHubDimitrios Bariamis
 		cc16b24b17 Update Flashinfer to 0.6.6 (#36768) 
Signed-off-by: Dimitrios Bariamis <12195802+dbari@users.noreply.github.com>
Co-authored-by: Dimitrios Bariamis <12195802+dbari@users.noreply.github.com>
 2026-03-12 13:19:19 -04:00
Amr MahdiandGitHub 12b38c0f45 [CI/Build] Allow mounting AWS credentials for sccache S3 auth (#35912) 
Signed-off-by: Amr Mahdi <amrmahdi@meta.com>
 2026-03-03 14:30:47 -08:00
Tyler Michael SmithGitHubClaude Opus 4.6
 		eb19955c37 [WideEP] Remove pplx all2all backend (#33724) 
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-02-26 14:30:10 -08:00
Seungmin KimGitHub88plugMichael Goin
 		160424a937 [Bugfix] Fix CUDA compatibility path setting for both datacenter and consumer NVIDIA GPUs (#33992) 
Signed-off-by: Seungmin Kim <8457324+ehfd@users.noreply.github.com>
Signed-off-by: Andrew Mello <19512127+88plug@users.noreply.github.com>
Co-authored-by: 88plug <19512127+88plug@users.noreply.github.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2026-02-25 18:15:51 -08:00
Wei ZhaoGitHubCyrus Leung
 		ea5f903f80 Bump Flashinfer Version and Re-enable DeepSeek NVFP4 AR+Norm Fusion (#34899) 
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-02-20 13:37:31 -08:00
zifeitongGitHubWentao Ye
 		52181baaea Update DeepGEMM version pin in Dockerfile to match #32479 (#33935) 
Signed-off-by: Zifei Tong <zifeitong@gmail.com>
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2026-02-07 05:30:22 -08:00
Dimitrios BariamisGitHubDimitrios Bariamismgoin
 		207c3a0c20 Fix RoutingMethodType logic (#33919) 
Signed-off-by: Dimitrios Bariamis <12195802+dbari@users.noreply.github.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Dimitrios Bariamis <12195802+dbari@users.noreply.github.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2026-02-06 14:03:34 -08:00
杨朱 · KikiGitHubClaude Opus 4.5
 		a0a984ac2e [CI/Build] Remove hardcoded America/Los_Angeles timezone from Dockerfiles (#33553) 
Signed-off-by: carlory <baofa.fan@daocloud.io>
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
 2026-02-02 22:32:39 -08:00
Dimitrios BariamisGitHubDimitrios BariamisCyrus Leung
 		f0bca83ee4 Add support for Mistral Large 3 inference with Flashinfer MoE (#33174) 
Signed-off-by: Dimitrios Bariamis <12195802+dbari@users.noreply.github.com>
Co-authored-by: Dimitrios Bariamis <12195802+dbari@users.noreply.github.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-01-30 22:48:27 -08:00
Pengchao WangGitHubYe Qi
 		2515bbd027 [CI/Build][BugFix] fix cuda/compat loading order issue in docker build (#33116) 
Signed-off-by: Pengchao Wang <wpc@fb.com>
Co-authored-by: Ye (Charlotte) Qi <yeq@meta.com>
 2026-01-29 00:19:05 -08:00
Orion Reblitz-RichardsonGitHubKevin H. Luu
 		68b0a6c1ba [CI][torch nightlies] Use main Dockerfile with flags for nightly torch tests (#30443) 
Signed-off-by: Orion Reblitz-Richardson <orionr@meta.com>
Signed-off-by: Orion Reblitz-Richardson <orionr@gmail.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
 2026-01-23 10:22:56 -08:00
elvischenvandGitHub 808d6fd7b9 Bump Flashinfer to v0.6.1 (#30993) 
Signed-off-by: elvischenv <219235043+elvischenv@users.noreply.github.com>
 2026-01-21 08:49:50 -08:00
Mritunjay Kumar SharmaandGitHub 9e078d0582 [CI/Build][Docker] Add centralized version manifest for Docker builds (#31492) 
Signed-off-by: Mritunjay Sharma <mritunjay.sharma@chainguard.dev>
 2026-01-17 13:45:30 +00:00
emricksini-handGitHub 2a60ac91d0 [Improvement] Persist CUDA compat libraries paths to prevent reset on apt-get (#30784) 
Signed-off-by: emricksini-h <emrick.birivoutin@hcompany.ai>
 2026-01-13 14:35:05 -08:00
Shang WangandGitHub 33156f56e0 [docker] A follow-up patch to fix #30913: [docker] install cuda13 version of lmcache and nixl (#31775) 
Signed-off-by: Shang Wang <shangw@nvidia.com>
 2026-01-07 23:47:02 -08:00
Seiji EicherGitHubCyrus Leung
 		3c98c2d21b [CI/Build] Allow user to configure NVSHMEM version via ENV or command line (#30732) 
Signed-off-by: Seiji Eicher <seiji@anyscale.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-01-05 15:56:08 -08:00
Qidong SuandGitHub af1b07b0c5 [docker] install cuda13 version of lmcache and nixl (#30913) 
Signed-off-by: Qidong Su <soodoshll@gmail.com>
 2026-01-05 12:50:39 -08:00
Nick CaoandGitHub d7e05ac743 [docker] Fix downloading sccache on aarch64 platform (#30070) 
Signed-off-by: Nick Cao <nickcao@nichi.co>
 2025-12-23 21:36:33 -08:00
Amr MahdiandGitHub c0a88df7f7 [docker] Allow kv_connectors install to fail on arm64 (#30806) 
Signed-off-by: Amr Mahdi <amrmahdi@meta.com>
 2025-12-16 16:41:57 -08:00
Amr MahdiandGitHub ff21a0fc85 [docker] Restructure Dockerfile for more efficient and cache-friendly builds (#30626) 
Signed-off-by: Amr Mahdi <amrmahdi@meta.com>
 2025-12-15 18:52:19 -08:00
Noa NeriaandGitHub 6366c098d7 Validating Runai Model Streamer Integration with S3 Object Storage (#29320) 
Signed-off-by: Noa Neria <noa@run.ai>
 2025-12-04 18:04:43 +08:00
Shengqi ChenandGitHub 1109f98288 [CI] fix docker image build by specifying merge-base commit id when downloading pre-compiled wheels (#29930) 
Signed-off-by: Shengqi Chen <harry-chen@outlook.com>
 2025-12-03 14:08:19 -08:00
Amr MahdiandGitHub f5d3d93c40 [docker] Build CUDA kernels in separate Docker stage for faster rebuilds (#29452) 
Signed-off-by: Amr Mahdi <amrmahdi@meta.com>
 2025-12-03 11:41:53 +00:00
Benjamin BartelsandGitHub 2d613de9ae [CI/Build] Fixes missing runtime dependencies (#29822) 
Signed-off-by: bbartels <benjamin@bartels.dev>
 2025-12-02 10:21:49 -08:00
Andrii SkliarGitHubAndrii Skliar
 		a5345bf49d [BugFix] Fix plan API Mismatch when using latest FlashInfer (#29426) 
Signed-off-by: Andrii Skliar <askliar@askliar-mlt.client.nvidia.com>
Co-authored-by: Andrii Skliar <askliar@askliar-mlt.client.nvidia.com>
 2025-11-27 11:34:59 -08:00
AlecandGitHub c4c0354eec [CI/Build] allow user modify pplx and deepep ref by ENV or command line (#29131) 
Signed-off-by: alec-flowers <aflowers@nvidia.com>
 2025-11-26 17:41:16 +00:00
汪志鹏andGitHub 7012d8b45e [Docker] Optimize Dockerfile: consolidate apt-get and reduce image size by ~200MB (#29060) 
Signed-off-by: princepride <wangzhipeng628@gmail.com>
 2025-11-24 19:54:00 -07:00
Benjamin BartelsandGitHub 4d6afcaddc [CI/Build] Moves to cuda-base runtime image while retaining minimal JIT dependencies (#29270) 
Signed-off-by: bbartels <benjamin@bartels.dev>
Signed-off-by: Benjamin Bartels <benjamin@bartels.dev>
 2025-11-24 11:40:54 -08:00
Benjamin BartelsandGitHub eb5352a770 [CI/build] Removes source compilation from runtime image (#26966) 
Signed-off-by: bbartels <benjamin@bartels.dev>
 2025-11-22 10:23:09 -08:00
Cyrus LeungandGitHub 9452863088 Revert "Revert #28875 (#29159)" (#29179) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-11-21 04:27:43 -08:00
Cyrus LeungandGitHub 4d7231e774 Revert #28875 (#29159) 2025-11-21 01:40:17 -08:00
Qidong SuGitHubCyrus Leung
 		698024ecce [Doc] update installation guide regarding aarch64+cuda pytorch build (#28875) 
Signed-off-by: Qidong Su <soodoshll@gmail.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2025-11-20 19:40:25 -08:00