karylab_agents/vllm
1
0
Fork 0
forked from Karylab-cklius/vllm
Code Pull Requests 2 Actions 1 Packages Activity
15,342 Commits 354 Branches 175 Tags
d7ccecd2b7c96c274d19b1892905311c1947e910
Commit Graph
15342 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Woosuk Kwon d7ccecd2b7 Enable PW CUDA graphs 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
 2026-04-06 00:35:01 +00:00
Woosuk Kwon 1fed50d74f silu_mul_quant_fp4 fusion 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
 2026-04-03 01:05:18 +00:00
Woosuk Kwon f9bf662e5b fuse parallel linears 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
 2026-04-01 05:47:41 +00:00
Woosuk Kwon 14e2241f77 remove torch compile decorator 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
 2026-03-31 21:06:51 +00:00
Woosuk Kwon cdd23258cf dummy loader hack 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
 2026-03-31 19:25:36 +00:00
Woosuk Kwon cec6774e9b registry 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
 2026-03-31 19:23:24 +00:00
Woosuk Kwon 355be167e6 fuse & parallelize linears 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
 2026-03-31 06:30:37 +00:00
Woosuk Kwon d67e21b26e fuse moe scale_and_add 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
 2026-03-31 04:47:26 +00:00
Woosuk Kwon 6a2c13a6f0 fuse concat + fp8 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
 2026-03-31 03:59:16 +00:00
Woosuk Kwon b443e6702e fuse mla cache 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
 2026-03-31 03:38:42 +00:00
Woosuk Kwon 34d73a3375 Fuse indexer k cache update 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
 2026-03-30 22:28:33 +00:00
Woosuk Kwon 9d7beab915 indexer_k_quant_triton 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
 2026-03-30 19:47:36 +00:00
Woosuk Kwon f2ecfa9cd7 topk init 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
 2026-03-30 17:12:27 +00:00
Woosuk Kwon e4cdaf199d minor 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
 2026-03-30 17:02:23 +00:00
Woosuk Kwon 2b72935629 inline indexer 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
 2026-03-30 16:47:09 +00:00
Woosuk Kwon 1903df8328 all reduce + residual + rms norm 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
 2026-03-30 05:28:13 +00:00
Woosuk Kwon d872b0a082 fix 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
 2026-03-30 04:01:19 +00:00
Woosuk Kwon 84deceffb7 remove attn 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
 2026-03-30 03:07:54 +00:00
Woosuk Kwon e269b614c0 fuse quant & indexer weight update 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
 2026-03-30 03:04:25 +00:00
Woosuk Kwon 24090c52f3 adjust fused_q grid 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
 2026-03-30 02:31:20 +00:00
Woosuk Kwon 063fd29c98 fused op 1 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
 2026-03-30 01:49:10 +00:00
Woosuk Kwon 156e12ba35 indexer rope 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
 2026-03-29 23:11:34 +00:00
Woosuk Kwon 3e5c06dd7d rope 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
 2026-03-29 22:56:32 +00:00
Woosuk Kwon cc08dad785 layernorm 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
 2026-03-29 19:26:57 +00:00
Woosuk Kwon 976293e374 wip 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
 2026-03-29 17:26:23 +00:00
Woosuk Kwon 6efd919548 wip 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
 2026-03-29 17:19:31 +00:00
Woosuk Kwon 2145abaade Merge branch 'main' into woosuk/ds-exp 2026-03-27 22:23:32 +00:00
Giancarlo DelfinandGitHub 384e4d5f48 [Model Runner V2] Rebuild attention metadata before eagle decode full… (#38311) 
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
 2026-03-27 13:46:42 -07:00
Nicolò LucchesiandGitHub 44a6528028 [CI] Skip failing test (#38369) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2026-03-27 13:25:19 -07:00
Kyle SayersandGitHub 648edcf729 [QeRL] Compose online quantization with quantized reloading (#38032) 
Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>
 2026-03-27 13:22:33 -07:00
Michael GoinGitHubClaude
 		7ba425e916 Add short flag -sc for --speculative-config argument (#38380) 
Co-authored-by: Claude <noreply@anthropic.com>
 2026-03-27 12:04:22 -07:00
Gregory ShtrasbergandGitHub b8665383df [ROCm] Fix GPT-OSS import for triton 3.6 (#37453) 
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
 2026-03-27 18:00:57 +00:00
Rohan PotdarGitHubAndreas Karatzas
 		0e9358c11d {ROCm]: gpt-oss fusion/padding fixes (#38043) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com>
Signed-off-by: Rohan Potdar <66227218+Rohan138@users.noreply.github.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-03-27 12:19:15 -04:00
Harry MellorGitHubgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 		21d2b53f88 Remove need for explicit \n in docstring lists for --help formatting (#38350) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-03-27 08:38:00 -07:00
Jonas M. KüblerandGitHub 98e7f223b9 enable skipping of SW attention layers when using FP8 KV cache (#33695) 
Signed-off-by: Jonas Kuebler <kuebj@amazon.com>
 2026-03-27 07:25:02 -06:00
Juan Pérez de AlgabaGitHubRussell Bryant
 		b111f8a61f fix(security): Add VLLM_MAX_N_SEQUENCES environment variable and enforce limit (#37952) 
Signed-off-by: jperezde <jperezde@redhat.com>
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Russell Bryant <rbryant@redhat.com>
 2026-03-27 09:02:10 -04:00
Sage MooreandGitHub 497e234d38 [EPLB] Cleanup the transfer logic for the various eplb maps (#34520) 
Signed-off-by: Sage Moore <sagmoore@redhat.com>
Signed-off-by: Sage Moore <sage@neuralmagic.com>
 2026-03-27 10:18:46 +01:00
dtcandGitHub 6287e7fa20 [P/D] Mooncake: Add unit tests and minor fixes for mooncake connector (#36946) 
Signed-off-by: Tianchen Ding <dtcccc@linux.alibaba.com>
 2026-03-27 09:26:40 +01:00
Shengqi ChenGitHubClaude Opus 4.6gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>Copilot
 		84e439a9cb [CI/Build] Move nightly wheel index generation to a single post-build step (#38322) 
Signed-off-by: Shengqi Chen <harry-chen@outlook.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
 2026-03-27 07:44:18 +00:00
Yuichiro UtsumiGitHubgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 		a1746ff9ec [Doc] Clarify Helm chart location in deployment guide (#38328) 
Signed-off-by: Yuichiro Utsumi <utsumi.yuichiro@fujitsu.com>
Signed-off-by: Yuichiro Utsumi <81412151+utsumi-fj@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-03-27 15:43:02 +08:00
Flora FengandGitHub aee4c14689 [Bugfix] Fix Hermes tool parser when stream interval > 1 (#38168) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2026-03-27 14:42:26 +08:00
Bowen BaoandGitHub 0ae89f18fd [Refactor] Move FusedMoE hidden_size roundup to quant_method (#34285) 
Signed-off-by: Bowen Bao <bowenbao@amd.com>
 2026-03-26 23:38:26 -07:00
wenjun liuandGitHub c2b17d71af [CI] Add xpu auto-label rule for Intel GPU/XPU PRs (#38320) 
Signed-off-by: wendyliu235 <wenjun.liu@intel.com>
 2026-03-27 14:22:38 +08:00
Li, JiangandGitHub becaed6ec8 [CPU] Support CT W4A16 on CPU MP kernel (#38219) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2026-03-27 14:15:28 +08:00
Xiaoshuang WangandGitHub a8eab8f30d [Model] Extract GatedDeltaNetAttention into shared layer for Qwen3Next and Qwen3.5 (#37975) 
Signed-off-by: wxsIcey <1790571317@qq.com>
Signed-off-by: Icey <1790571317@qq.com>
 2026-03-27 14:13:21 +08:00
cjackalandGitHub 2babac0bed [frontend] dump openai responses type by alias (#38262) 
Signed-off-by: cjackal <44624812+cjackal@users.noreply.github.com>
 2026-03-27 05:58:20 +00:00
Or OzeriandGitHub 7cc302dd87 [kv_offload+HMA][7/N]: Support register_kv_caches for hybrid models (#37853) 
Signed-off-by: Or Ozeri <oro@il.ibm.com>
 2026-03-27 08:38:33 +03:00
Woosuk Kwon a17a1f12dc inlined dsv3.2 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
 2026-03-27 05:20:23 +00:00
BviciiGitHubCyrus Leung
 		999dfc1622 [Bugfix] Offload blocking tokenizer ops to shared thread pool to unblock event loop (#34789) 
Signed-off-by: Bvicii <yizhanhuang2002@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-03-26 22:17:00 -07:00
wenjun liuandGitHub d86060122a [CI/Build] enable Intel XPU test flow with prebuilt image (#37447) 
Signed-off-by: wendyliu235 <wenjun.liu@intel.com>
 2026-03-26 18:16:04 -07:00