karylab_agents/vllm
1
0
Fork 0
forked from Karylab-cklius/vllm
Code Pull Requests 2 Actions 2 Packages Activity
19,249 Commits 354 Branches 175 Tags
59c62cd2b4214dacb00a8efdfbee7af69058c1cf
Commit Graph
19249 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Wentao YeandGitHub 59c62cd2b4 Merge branch 'main' into wentao-support-rms-norm-uncontiguous 
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2026-07-28 14:31:16 -04:00
Shanshan ShenandGitHub 6c7e679f04 [ROCm][Bugfix] Sanitize AITER paged-MQA logits before sparse top-k for DeepSeek-V4 (#49714) 
Signed-off-by: shen-shanshan <467638484@qq.com>
 2026-07-28 11:08:50 -07:00
labAxiaomingandGitHub 1db989bbf1 [Bugfix][Multimodal] Fix video temporal padding estimates (#49030) 
Signed-off-by: xiaoming <1259730330@qq.com>
 2026-07-29 01:57:31 +08:00
Brian DellabettaandGitHub 8a7b3c2990 [compressed-tensors] update find_matched_target order to prioritize fused name matches over class match (#49483) 
Signed-off-by: Brian Dellabetta <bdellabe@redhat.com>
 2026-07-28 17:03:08 +00:00
Andreas KaratzasandGitHub 05a0814863 [ROCm] Fix and optimize GPT-J-style MRoPE (#49906) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-07-28 10:50:10 -06:00
jiacao-amdGitHubTJian
 		4f56321d7e [ROCm] Cache fp32 upcast of static e8m0 weight scale in AITER scaled_mm (#47773) 
Signed-off-by: jiacao-amd <jiahui.cao@amd.com>
Co-authored-by: TJian <tunjian.tan@embeddedllm.com>
 2026-07-28 11:34:16 -05:00
Bugen ZhaoandGitHub 01661cc57f [Rust][Benchmark] Make vllm bench serve Rust delegation opt-in (#50081) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-07-28 16:21:12 +00:00
Yiliu DongGitHubOpenAI Codex
 		ba702e978e [Attention] Skip sparse indexer scoring for dense short prefills (#48407) 
Signed-off-by: Yiliu Dong <91178480+qianlihuang@users.noreply.github.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-07-28 16:17:22 +00:00
Nick HillGitHubAndreas Karatzas
 		6453fc0b8c [Bugfix] Don't reuse engine core payload buffer while zmq is sending it (#50053) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-07-28 16:09:28 +00:00
IBRAHIM IBRAHIMGitHubibrahimibrahimClaude Opus 4.8
 		4fb483ca86 [Docs] Expand llm-d integration page (#45432) 
Signed-off-by: ibrahimibrahim <ibib2595@gmail.com>
Co-authored-by: ibrahimibrahim <ibib2595@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-28 15:47:56 +00:00
Itay EtelisGitHubItay EtelisItay Etelis
 		30217b0e80 [Bugfix][KV Offload][P2P] Scope serve state to fetch rounds (#49877) 
Signed-off-by: Itay Etelis <itay.etelis@ibm.com>
Co-authored-by: Itay Etelis <itay.etelis@ibm.com>
Co-authored-by: Itay Etelis <Itay.etelis@gmail.com>
 2026-07-28 18:01:55 +03:00
Nick HillandGitHub 0d0504b54c [Core] Warm up runner-owned Triton kernels before the first request (#49903) 2026-07-28 07:58:02 -07:00
Jonguk CheongGitHubOpenAI Codex
 		1e81853afc [Bugfix][KV Offload] Keep Mamba block span unscaled under DCP (#49964) 
Signed-off-by: Jonguk Cheong <jdal3031@snu.ac.kr>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-07-28 17:41:55 +03:00
oops-oomGitHuboops-oomClaude Opus 4.8Wentao YeShengqi Chen
 		b6cbba8bc8 [Bugfix][Kernel] Fix batch invariance in RMSNorm kernels by pinning block size (#48391) 
Signed-off-by: oops-oom <73481342@qq.com>
Signed-off-by: oops-oom <liubin8905@vip.qq.com>
Co-authored-by: oops-oom <73481342@qq.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: Shengqi Chen <harry-chen@outlook.com>
 2026-07-28 22:24:02 +08:00
Nick HillGitHubClaude Opus 4.8
 		94100b5915 [CI] Wire untethered test files into CI jobs (#49340) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-28 14:02:31 +00:00
Harry MellorandGitHub 62d8db7c05 [Bugfix] Add missing vllm/models/kimi_k3/__init__.py (#50131) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-07-28 13:47:04 +00:00
Nick HillGitHubClaude Fable 5
 		601fa9a74e [KV Connector] Support NIXL heterogeneous P/D block sizes for hybrid models (#49612) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-07-28 06:45:02 -07:00
liangel-02GitHubmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Matthew BonanniMichael Goin
 		9b9fc4039c add epilogue hook to flex attention (#45841) 
Signed-off-by: Angel Li <liangel@meta.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2026-07-28 06:26:55 -07:00
Nicolò LucchesiandGitHub 98e91a9600 [PD][NixlPush] Skip extra add_remote_agent step in D->P handshake (#49345) 
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
 2026-07-28 14:57:12 +02:00
Xin HeGitHubKunshang Ji
 		948107acf7 [Bugfix] Enhance extra_config handling for layer name suffix matching (#48589) 
Signed-off-by: Xin He <xin3.he@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-07-28 20:37:55 +08:00
Itay AlroyandGitHub 35efdf6b34 [Elastic EP] Async preparation (#47288) 
Signed-off-by: Itay Alroy <ialroy@nvidia.com>
 2026-07-28 05:19:13 -07:00
Kevin H. LuuGitHubOpenAI Codex
 		d2bfc6fe20 [Build] Fix DeepEP CUDA driver stub linking (#50103) 
Signed-off-by: khluu <khluu000@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-07-28 12:09:19 +00:00
ReidandGitHub 912d6b619d [Rust Frontend] Align sampling validation with Python (#47494) 
Signed-off-by: reidliu41 <reid201711@gmail.com>
 2026-07-28 12:02:00 +00:00
ReidandGitHub bf9f23003c [Rust Frontend] Fix finish reason for named tool choices (#49496) 
Signed-off-by: reidliu41 <reid201711@gmail.com>
 2026-07-28 10:53:18 +00:00
Jiangyun ZhuGitHubOpenAI Codexmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		25ace8fe5d [CI] Increase Qwen3.5 MTP GSM8K generation length (#49881) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-07-28 18:04:36 +08:00
Chris LeonardGitHubShengqi Chen
 		247470f23a [CI] Add PyTorch stable ABI audit check (#48164) 
Signed-off-by: Chris Leonard <chleonar@redhat.com>
Co-authored-by: Shengqi Chen <harry-chen@outlook.com>
 2026-07-28 02:37:01 -07:00
Liangliang MaGitHubmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		88402a41c4 [Test] Skip ROCm AITER MLA prefill tests on non-ROCm platforms (#49945) 
Signed-off-by: Liangliang-Ma <liangliang.ma@intel.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-07-28 16:26:49 +08:00
Bugen ZhaoGitHubOpenAI Codex
 		5ed3faa43d [Rust Frontend] Add ordinary-text tokenizer encoding (#49992) 
Co-authored-by: OpenAI Codex <noreply@openai.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-07-28 15:46:51 +08:00
Thien TranandGitHub 61ac368021 [Kimi-K3] Add AttnRes kernels (#50090) 
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
 2026-07-28 00:22:09 -07:00
Andreas KaratzasGitHubOpenAI Codex
 		99b57a4823 [CI][ROCm] Soft fail LoRA mirror (#50086) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-07-28 00:18:04 -07:00
aoshen02GitHubaoshen02vx120Claude Opus 5Roger Wang
 		b09688a6e7 [Bugfix][Spec Decode] Preserve draft buffers across level-2 sleep (#49774) 
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Signed-off-by: vx120 <893600387@qq.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: vx120 <893600387@qq.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
 2026-07-28 00:16:21 -07:00
ChaunceyGitHubAndreas Karatzas
 		03a2d03367 [Bugfix] Respect cgroup memory limits on all platforms (#49966) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-07-28 14:38:20 +08:00
Shuolei WangandGitHub 9069a57139 [Core][Frontend] Add weight version tagging for RL rollouts (#49040) 
Signed-off-by: Shuolei Wang <shuoleiwang123@gmail.com>
Signed-off-by: Shuolei Wang <948904026@qq.com>
 2026-07-28 14:35:29 +08:00
Li, JiangandGitHub 90245f4190 [Bugfix] Fix multi-modal support on CPU MRV2 (#50073) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2026-07-28 06:09:48 +00:00
afriedriandGitHub f472ab0a4c Remove triton per group quant [ROCm] [Bugfix] (#49621) 
Signed-off-by: Andy Friedrich <afriedri@amd.com>
 2026-07-28 05:53:46 +00:00
Ayushman SinghandGitHub 74587939b1 [Build] Fix CUDA arch detection producing kernel-less builds on SM121 (#49904) 2026-07-27 22:01:56 -07:00
Nick HillandGitHub d223c900d8 [Bugfix] Only pad transformers backend value when it is narrower (#50060) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-07-27 23:36:45 -05:00
MINJUN GILGitHubOpenAI Codex
 		52c3c4a42f [Bugfix][KV Offload][OBJ] Preserve job completion during cleanup (#49947) 
Signed-off-by: MINJUN GIL <alswnsrlf12@naver.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-07-28 07:10:52 +03:00
Chang GuoGitHubGPT-5.6 SolOr Ozeri
 		a8f296083f [KV Offload] Make compact secondary identity TP-independent (#49858) 
Signed-off-by: Change72 <changg@nvidia.com>
Co-authored-by: GPT-5.6 Sol <noreply@openai.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
 2026-07-28 07:10:20 +03:00
Colin ZGitHubAndreas Karatzas
 		fbb1ef6803 [Bugfix] Fix DeepseekV4FP8 Quark MXFP4 crash on list-valued weight (#49634) 
Signed-off-by: Colin Zeng <Colin.Zeng@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-07-27 22:32:42 -05:00
fxmartyGitHubFelix MartyAndreas Karatzas
 		33fe71a4d3 [AMD] Revert Mxfp4MoeBackend.TRITON_UNFUSED fallback (#46491) 
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
Co-authored-by: Felix Marty <Felix.Marty@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-07-27 22:31:23 -05:00
Varun Sundar RabindranathGitHubVarun Sundar Rabindranath
 		d18ed2304a [KV-offload][FS] : Batch store/load_block in C (#49152) 
Signed-off-by: <>
Co-authored-by: Varun Sundar Rabindranath <varun-sundar-rabindranath@h100-01.nemg-001.lab.rdu2.dc.redhat.com>
 2026-07-28 11:21:53 +08:00
nvbfalkandGitHub 60915c972c [Feature] Add VidCom2 video token pruning (#47750) 
Signed-off-by: Benedikt Falk <bfalk@nvidia.com>
 2026-07-28 03:20:17 +00:00
fxmarty-amdGitHubAndreas Karatzas
 		7aea73d83d [ROCm][Quark][6/N] Use MXFP4 linear kernel abstraction for aiter backend (#49348) 
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-07-27 21:35:57 -05:00
Yan MaandGitHub 73af7a362a [XPU] Add online fp8 quantization test (#44513) 
Signed-off-by: Yan Ma <yan.ma@intel.com>
 2026-07-28 10:34:33 +08:00
Andreas KaratzasandGitHub e68bfc2828 [CI][ROCm] Soft-fail Python-only installation mirror (#50041) 
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
 2026-07-27 19:22:53 -07:00
frida-anderssonandGitHub 02b6ecf07c [ROCm][DSv3.2] Eliminate per-decode FillFunctor launches in sparse-MLA hot loop (#44527) 
Signed-off-by: Frida Andersson <fanderss@amd.com>
 2026-07-28 10:12:20 +08:00
avininjamay8GitHubavininjamay8avininjamay8
 		1206891822 [ROCm][KVConnector][MoRI-IO] Fix WRITE-mode remote-TP rank collapse (#46332 follow-up) (#47764) 
Signed-off-by: avininjamay8 <avininjamay8@users.noreply.github.com>
Signed-off-by: avininjamay8 <Avinash.Paul@amd.com>
Co-authored-by: avininjamay8 <avininjamay8@users.noreply.github.com>
Co-authored-by: avininjamay8 <avpaul@amd.com>
 2026-07-27 20:26:24 -05:00
Woosuk KwonandGitHub 60b3d39cd3 [Docs] Remove experimental warning for EP (#50057) 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
 2026-07-27 17:06:28 -07:00
Lucas WilkinsonandGitHub 60417b4b74 [Core][PCP] Select MRV2 when PCP is enabled (#50034) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2026-07-27 17:03:26 -07:00