karylab_agents/vllm
1
0
Fork 0
forked from Karylab-cklius/vllm
Code Pull Requests 2 Actions 1 Packages Activity
17,191 Commits 354 Branches 175 Tags
dependabot/github_actions/actions/setup-python-6.2.0
Commit Graph
17191 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Gregory ShtrasbergGitHubRohan138
 		e724b0ea8d [ROCm] ROCm7.2.2 + profiler fix + AITER 0.1.12.post2 (#41386) 
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com>
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
Co-authored-by: Rohan138 <rohanpotdar138@gmail.com>
 2026-05-04 13:07:19 -05:00
Keyi LiGitHubKeyi LiClaudeFlora Feng
 		712ad0286c [Bugfix] KimiK2ReasoningParser: guard against buffered end-token in streaming (#41068) 
Signed-off-by: Keyi Li <likey6688@gmail.com>
Co-authored-by: Keyi Li <likey6688@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-05-04 17:42:05 +00:00
Ekagra RanjanandGitHub 321fa2d6d1 Limit gpu utils and lower max BS on test_transcription_api_correctness.py (#41649) 
Signed-off-by: Ekagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com>
 2026-05-04 10:30:02 -07:00
Wentao YeandGitHub 3e1ad4435f [Bug] Fix tests/compile/test_config.py AttributeError: 'NoneType' object has no attribute 'dtype' (#41288) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-05-05 00:22:07 +08:00
Netanel HaberandGitHub 8decbfa02c Test nemotron nano-v2 and nemotron nano-v3 separately, disable super-omni redundant tests (#41616) 
Signed-off-by: Netanel Haber <58652339+netanel-haber@users.noreply.github.com>
 2026-05-04 16:31:37 +03:00
Stefano CastagnettaandGitHub 62ba7516e8 Revert "[Doc] Fix RTD build: pytorch.org/docs/stable/objects.inv returns 404" (#41618) 
Signed-off-by: Stefano Castagnetta <scastagnetta@nvidia.com>
 2026-05-04 04:47:42 -07:00
Stefano CastagnettaandGitHub 6f53753fc9 [Bugfix] Apply ruff-format to hyperclovax.py (#41620) 
Signed-off-by: Stefano Castagnetta <scastagnetta@nvidia.com>
 2026-05-04 03:37:16 -07:00
Andreas KaratzasandGitHub 6ec9bbec38 [CI] Stabilize cpu offload compressed tensors test (#41102) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-05-04 05:22:42 +00:00
Andreas KaratzasandGitHub 01d4d1ad37 [ROCm][CI] Align spec decode logprob test prefill settings (#41335) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-05-04 04:33:29 +00:00
Fang HanGitHubClaude Opus 4.7
 		c103c02a1a [Transformers v5] Vendor HCXVisionConfig for compatibility (#38447) 
Signed-off-by: Fang Han <fhan0520@gmail.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-04 04:19:52 +00:00
Andreas KaratzasandGitHub 67058ca326 [CI] Clean up remote servers on pytest parent exit (#41570) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-05-04 03:11:22 +00:00
Akim TsvigunandGitHub 894a02500b [Bench] Forward --seed to CustomDataset and CustomMMDataset shuffle (#40788) 
Signed-off-by: akimtsvigun <akimtsvigun@gmail.com>
 2026-05-04 00:39:10 +00:00
David OyGitHubDavid OyClaudeVadim Gimpelson
 		66dfee7121 [Bugfix] Fix degenerate KV cache stride causing TMA cudaErrorIllegalInstruction (#40737) 
Signed-off-by: David Oy <david@baseten.co>
Signed-off-by: David Oy <58150256+the-david-oy@users.noreply.github.com>
Signed-off-by: David Oy <david.oy@baseten.co>
Co-authored-by: David Oy <david@baseten.co>
Co-authored-by: Claude <claude@anthropic.com>
Co-authored-by: Vadim Gimpelson <156319763+vadiklyutiy@users.noreply.github.com>
 2026-05-03 23:52:18 +00:00
Alex BrooksandGitHub db9a84e0cd [Bugfix] Fix FP8 Bias Loading (#41424) 
Signed-off-by: Alex Brooks <albrooks@redhat.com>
 2026-05-03 20:30:04 +00:00
tomeras91andGitHub cb03fee32b [Bugfix][Ray] Fix RayExecutorV2 actor name collision with DP > 1 (#40398) 
Signed-off-by: Tomer Asida <57313761+tomeras91@users.noreply.github.com>
 2026-05-03 13:00:41 -07:00
Wei ZhaoandGitHub c51df43005 Disable flashinfer autotune temporarily due to correctness issues (#41524) 
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
 2026-05-03 16:19:59 +00:00
Taneem IbrahimandGitHub 54dc64d5d3 [Doc] Add Qwen3-30B-A3B-Thinking-2507-FP8 to batch invariance verified models (#41513) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
 2026-05-03 08:47:55 -04:00
Woosuk KwonandGitHub e6ff3e9c83 [MRV2] Add shutdown() method (#41297) 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
v0.20.2rc0 		2026-05-02 21:06:30 -07:00
Jinzhen LinandGitHub 08834cc3ce [Quantization] add humming mxfp4 moe backend (#41083) 
Signed-off-by: Jinzhen Lin <jinzhen.ljz@antgroup.com>
 2026-05-02 18:36:03 -07:00
Roger WangGitHubCopilot
 		856ec4804a [DSv4] Tune default value of VLLM_MULTI_STREAM_GEMM_TOKEN_THRESHOLD (#41526) 
Co-authored-by: Copilot <copilot@github.com>
 2026-05-02 18:32:09 -07:00
Yongye ZhuandGitHub 1c607d7b2c [DSV4] Guard megamoe flag with Pure TP (#41522) 
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
 2026-05-02 16:41:40 -07:00
Michael GoinGitHubClaude Opus 4.7
 		4f7309fcc0 [CI] Add ci-fetch-log.sh helper for Buildkite job logs (#41517) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-02 15:23:59 -07:00
Michael GoinandGitHub 0a9362d6ab Revert "[Build] Make bundled DeepGEMM wheel portable across Python versions" (#41512) 2026-05-02 09:42:41 -07:00
Michael GoinGitHubClaude
 		cfd2573f23 [Build] Switch CUDA 13.0 wheel builds to PyTorch manylinux_2_28 base (#41416) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-05-02 05:51:28 -07:00
Hoang NguyenGitHubClaudemergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Isotr0py
 		c3ad791e1a [Bugfix][Gemma 4] Clamp soft-token estimate to max_soft_tokens (#40796) 
Signed-off-by: Hoang Nguyen <118159510+hnt2601@users.noreply.github.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-05-02 06:34:59 +00:00
Matthew SantiagoandGitHub 8586369f61 Refactor Step3Text loading to use AutoWeightsLoader (#41492) 
Signed-off-by: Matthew Santiago <carag.matthew@gmail.com>
 2026-05-02 06:22:14 +00:00
ChaunceyandGitHub ae3b4deb8a [Doc] Add Codex usage example (#41358) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2026-05-01 22:27:43 -07:00
Rita BrugarolasandGitHub c293ccc58e [ROCm][Bugfix] Fix init-time bias dtype cast when gate.out_dtype is None (#41405) 
Signed-off-by: Rita Brugarolas Brufau <rita.brugarolasbrufau@amd.com>
 2026-05-02 00:13:15 -04:00
Luka GovedičandGitHub d58c42e19c [vLLM IR] 2/N fused_add_rms_norm and maybe_inplace overload (#36823) 
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Signed-off-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
 2026-05-01 23:41:15 -04:00
Ekagra RanjanandGitHub 3e49479c4b Limit concurrency on test_transcription_api_correctness.py (#41478) 
Signed-off-by: Ekagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com>
 2026-05-02 03:19:07 +00:00
John CalderonandGitHub 964a4bc2a5 [MM][CG] Support ViT CG for Qwen2.5-VL (#40830) 
Signed-off-by: John Calderon <jcalderon@nvidia.com>
 2026-05-02 11:10:14 +08:00
FredericOdermattandGitHub c408fdd663 [Fix] Sync gemma4 chat template from hf (#39570) 
Signed-off-by: Frederic Odermatt <frederic.odermatt@44ai.ch>
 2026-05-02 03:06:54 +00:00
Andy LoandGitHub 5737770c6c Re-enable allreduce rms fusion for DP / PP (#41458) 
Signed-off-by: Andy Lo <andy@mistral.ai>
 2026-05-01 19:01:37 -04:00
Michael GoinandGitHub 0c99629ede [Build] Make bundled DeepGEMM wheel portable across Python versions (#41476) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-05-01 14:45:03 -07:00
Yongye ZhuandGitHub edd60ac93a [Bugfix] Fix persistent_topk inter-CTA init race on RadixRowState (#41444) 
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
 2026-05-01 14:42:52 -07:00
Yongye ZhuandGitHub bcf5cac9fb [DSV4] Add knob to enable pre-attn gemm (#41443) 
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
 2026-05-01 12:23:17 -07:00
Isotr0pyGitHubRoger Wang
 		a9484dac7b [Perf] Intergrate Tile Kernels head_compute_mix_kernel for Deepseek-V4 (#41255) 
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
 2026-05-01 15:01:17 -04:00
Matthew BonanniGitHubMichael GoinLucas Wilkinsonmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		f3fef12350 [Attention] Abstract the MLA prefill backends and eliminate cuDNN (#32623) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-05-01 13:36:20 -04:00
Wentao YeGitHubNick Hill
 		51295793a2 [Model Runner V2] Add logprob_token_ids support (#40559) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
 2026-05-01 10:02:03 -07:00
Michael GoinandGitHub 3ccc1ff495 [Eval][CI] Add basic mrcr eval to tests/evals/ (#40164) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-05-01 12:00:38 -04:00
vllmellmGitHubRita Brugarolasjunkang1991Luka GovedičTJian
 		529c671e80 [ROCm][FEAT] AITER Fused Allreduce + RMSNorm (#37646) 
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
Signed-off-by: Rita Brugarolas Brufau <rita.brugarolasbrufau@amd.com>
Signed-off-by: junkang1991 <junkangchow@gmail.com>
Co-authored-by: Rita Brugarolas <Rita.BrugarolasBrufau@amd.com>
Co-authored-by: junkang1991 <junkangchow@gmail.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Co-authored-by: TJian <tunjian.tan@embeddedllm.com>
 2026-05-01 23:07:18 +08:00
PleaplusoneGitHubMatthew Wong
 		bc635fad23 [ROCm][Deepseek] dsv3.2 further optimization (#41217) 
Signed-off-by: ganyi <ygan@amd.com>
Signed-off-by: Matthew Wong <Matthew.Wong2@amd.com>
Co-authored-by: Matthew Wong <Matthew.Wong2@amd.com>
 2026-05-01 23:06:00 +09:00
Artem PerevedentsevGitHubgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 		c3e64696cd [Perf] Warmup forward_native sampler kernel (#41375) 
Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-05-01 18:04:11 +04:00
sungsoo haandGitHub 4f7bde572a [Kernel] Pack output and LSE in DCP A2A (#41160) 2026-05-01 09:01:17 -04:00
Or OzeriandGitHub 2fa1f8ec00 [kv_offload+HMA][13/N]: Enable HMA support (#41445) 
This is the final PR in a series to enables HMA support for the
offloading connector. The connector advertises `SupportsHMA`
and is validated with unit tests and e2e tests.

Signed-off-by: Or Ozeri <oro@il.ibm.com>
 2026-05-01 12:30:03 +01:00
raviguptaamdandGitHub 7075df79b3 [ROCm] Enable DBO (Dynamic Batch Optimization) on ROCm (#34726) 
Signed-off-by: raviguptaamd <ravi.gupta@amd.com>
 2026-05-01 09:18:30 +00:00
Yuyi AoandGitHub 0dbaf9daad Refractor longcat loading to use AutoWeightsLoader (#41448) 
Signed-off-by: George-ao <yuyiao772@gmail.com>
 2026-05-01 09:07:23 +00:00
Seiji EicherGitHubMarwan SarieddineClaude
 		a3ec4a35f5 [Bugfix][Metrics] Fix RayPrometheusMetric.labels() returning shared labeled child (#40840) 
When vLLM runs with Ray Prometheus `vllm:request_success{finished_reason=...}`
only ever increments the repetition bucket regardless of the request's actual finish
reason; stop, length, abort, and error stay at zero. Root cause was `labels()` mutated
the wrapped Ray metric's default tags in place and returned self, so every `.labels(...)`
call on a given wrapper returned the same object. 

Co-authored-by: Marwan Sarieddine <sarieddine.marwan@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
Signed-off-by: Marwan Sarieddine <sarieddine.marwan@gmail.com>
Signed-off-by: Seiji Eicher <seiji@anyscale.com>
 2026-05-01 08:43:39 +01:00
Andreas KaratzasandGitHub 32964e7700 [ROCm][CI] Upgraded UCX and RIXL (#41210) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-05-01 16:40:47 +09:00
Bugen ZhaoGitHubwang.yuqi
 		a07642667d [Bugfix] Pass reasoning parser kwargs to structured output (#41199) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-04-30 23:38:02 -07:00