karylab_agents/vllm
1
0
Fork 0
forked from Karylab-cklius/vllm
Code Pull Requests 2 Actions 1 Packages Activity
17,864 Commits 354 Branches 175 Tags
2a09d50034c19ef4ab8564cdb53ddf4bed29a323
Commit Graph
17864 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Li, JiangandGitHub 8760f972ca [CPU] Refine CPU attention frontend (#45391) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2026-06-14 19:26:54 -07:00
maobaolongGitHubLi, Jiang <jiang1.li@intel.com>
 		b675cb7d0f [Bugfix][CPU] Honor cgroup memory limit when computing KV cache size (#45086) 
Signed-off-by: baoloongmao <baoloongmao@tencent.com>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>
 2026-06-14 19:26:50 -07:00
Chaojun ZhangandGitHub 2725c84aae [XPU] Enable sequence parallel support for XPU (#38608) 
Signed-off-by: chaojun-zhang <chaojun.zhang@intel.com>
Signed-off-by: Chaojun Zhang <chaojun.zhang@intel.com>
Signed-off-by: Chaojun,Zhang <chaojun.zhang@intel.com>
 2026-06-14 19:26:46 -07:00
Noa NeriaandGitHub 1801fad0ba [Bugfix] Stream Llama4 weight loading to avoid host-OOM with copy-returning loaders (#44645) 
Signed-off-by: Noa Neria <nneria@nvidia.com>
 2026-06-14 19:23:44 -07:00
Ting SUNandGitHub 3d6ce816f0 [Bugfix][Model] Validate runai_streamer model_loader_extra_config (#45291) 
Signed-off-by: Ting Sun <suntcrick@gmail.com>
 2026-06-14 19:23:30 -07:00
Taneem IbrahimandGitHub 2c764c089a Added real /v1/embeddings support for messages + chat_template_kw (#45173) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
 2026-06-15 09:08:10 +08:00
Michael MaandGitHub c621af1690 [BugFix] Fix prompt_embeds for multimodal models (#45383) 
Signed-off-by: ruinan ma <r7ma3088@gmail.com>
 2026-06-14 01:44:56 -07:00
Roger WangandGitHub e2bf2b3d84 [Perf] Use bisect for mm feature lookup in model runner v2 (#45566) 
Signed-off-by: Roger Wang <hey@rogerw.io>
 2026-06-14 00:22:53 -07:00
Amanzhol SalykovandGitHub 725c3bc808 [ROCm][Perf] Enable W4A16 FlyDSL MoE (#44400) 
Signed-off-by: amd-asalykov <asalykov@amd.com>
Signed-off-by: Amanzhol Salykov <asalykov@amd.com>
 2026-06-14 00:14:39 -07:00
Marceli FylcekGitHubKunshang Ji
 		9548a1887f [XPU] Support int4 group_size=32 W4A16 MoE (#45136) 
Signed-off-by: Marceli Fylcek <marceli.fylcek@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-06-14 00:14:35 -07:00
Jeff (Junze) MaandGitHub 9fd737badc [Bugfix][DCP] Fix illegal memory access in DCP a2a decode under full CUDA graphs (#45487) 2026-06-14 00:14:31 -07:00
Ekagra RanjanGitHubgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>Harry MellorBenjamin Chislett
 		4ef4492e9b [V1][Spec Decode] Add Dynamic SD (#32374) 
Signed-off-by: Ekagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com>
Signed-off-by: Benjamin Chislett <chislett.ben@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com>
 2026-06-14 00:14:27 -07:00
Shengqi ChenGitHubXin LiShawRongChange72
 		78e7293bb1 [Build] Fix CUDA arch build coverage gaps (#45277) 
Signed-off-by: Shengqi Chen <harry-chen@outlook.com>
Co-authored-by: Xin Li <xinli-sw@users.noreply.github.com>
Co-authored-by: ShawRong <ShawRong@users.noreply.github.com>
Co-authored-by: Change72 <Change72@users.noreply.github.com>
 2026-06-13 22:09:20 -07:00
achyuthan.sGitHubShengqi Chen
 		54bbf51668 [Bugfix] nightly Docker images crash with ImportError: AnthropicOutputConfig since May 28 (#44795) 
Signed-off-by: achyuthan.s <113010327+Achyuthan-S@users.noreply.github.com>
Signed-off-by: Achyuthan S <achyuthan.sivasankar@gmail.com>
Signed-off-by: Achyuthan Sivasankar <achyuthan.sivasankar@gmail.com>
Co-authored-by: Shengqi Chen <harry-chen@outlook.com>
 2026-06-13 21:45:29 -07:00
Nick HillandGitHub cf027b86af [Core] Simplify MRV2 async output handling (#45442) 2026-06-13 18:15:36 -07:00
Wentao YeGitHubmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		71b961dd35 [Perf] SM90 cutlass fp8 mm supports odd M by swap_ab, 180~290% kernel performance improvement (#44572) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-06-13 12:05:45 -07:00
WEI CHENG CHIUGitHubClaude
 		521b88c29e [Bugfix] Reject structured outputs for diffusion decoders with a clear error (#45468) 
Signed-off-by: Wayne Chiu <waynehacking8@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-06-13 12:04:01 -07:00
Harry MellorandGitHub b3f0a0a0df Fix docs build on main (#45536) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-06-13 08:53:23 -07:00
Juan Pérez de AlgabaandGitHub 470229c37e [Security] Fix DoS via prompt_embeds on M-RoPE models (#45252) 
Signed-off-by: jperezde <jperezde@redhat.com>
 2026-06-13 10:17:38 +00:00
Juan Pérez de AlgabaGitHubHarry Mellor
 		2b3006076c [Security] Add timeout guard for regex compilation in structured outp… (#45118) 
Signed-off-by: jperezde <jperezde@redhat.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-06-13 09:52:56 +00:00
Wentao YeandGitHub 96fa5cdd9e [CI Bug] Fix ValueError: There is no module or parameter named 'model.vision_tower.vision_model' (#45478) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-06-13 02:38:37 -07:00
Andreas KaratzasandGitHub 9261dbbc55 Treat null completion max_tokens like the default (#45491) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-06-13 09:34:09 +00:00
Wentao YeandGitHub 2ecf7d0eb4 [Model Runner V2] Fix openai.InternalServerError: Error code: 500 - 'list index out of range' (#45467) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-06-13 01:44:16 -07:00
midasandGitHub 0d29612292 [Doc] Fix uv dependency resolution failure for setuptools during CPU source builds (x86 & ARM) (#45412) 
Signed-off-by: midas <the.anon.github@gmail.com>
 2026-06-13 06:18:58 +00:00
WEI CHENG CHIUandGitHub 5b2943f5a6 [Bugfix] Return the tokenizer from maybe_make_thread_pool so it survives pickling (#45460) 
Signed-off-by: Wayne Chiu <waynehacking8@gmail.com>
 2026-06-13 06:01:35 +00:00
Martin HickeyGitHubIsotr0py
 		43f0e024bc [Render] Add /derender endpoints for disaggregated postprocessing (#43606) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-06-13 13:55:33 +08:00
Andreas KaratzasandGitHub 1033ffac2e [CI] Wait for SSL cert refresher events in the test (#45489) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-06-13 04:57:18 +00:00
longguoGitHubClaudeRoger Wang
 		ff5a30cfac [Bugfix] Replace deprecated Qwen2VLImageProcessorFast with Qwen2VLImageProcessor (#42700) 
Signed-off-by: abinggo <107740309+abinggo@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
 2026-06-12 21:04:31 -07:00
WEI CHENG CHIUandGitHub 17ee5b1ac5 [Bugfix] Set type/role explicitly in streaming message_start event (#45376) 
Signed-off-by: Wayne Chiu <waynehacking8@gmail.com>
 2026-06-13 01:40:50 +00:00
Nick HillandGitHub 1a369783e9 [BugFix] Avoid prematurely freeing cached mm encoder outputs (#45347) 
Signed-off-by: Roger Wang <hey@rogerw.io>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-06-12 15:39:40 -07:00
Kevin H. LuuandGitHub e3e31e54b0 [Bugfix][CPU] Don't build triton-cpu on arm64 release image (#45401) 
Signed-off-by: khluu <khluu000@gmail.com>
 2026-06-12 14:51:45 -07:00
Fangzhou AiGitHubClaude Fable 5
 		badddd254f [ROCm][DSV4][Perf] Fuse inverse-RoPE and cache bf16 wo_a in o-projection (#45103) 
Signed-off-by: Fangzhou Ai <fangzhouai@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-06-12 15:57:09 -05:00
Michael GoinGitHubClaude
 		c90650088d Add the QuantizedActivation linear-kernel contract (#44260) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-06-12 13:48:15 -07:00
Michael GoinandGitHub 9eaacb23ec [Kernel] Consolidate Marlin thread-tile padding across all dense Marlin paths (#45295) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-06-12 13:46:21 -07:00
Wentao YeGitHubmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		78739c1946 [Model Runner v2] Migration from v1 to v2, with Qwen and DSv2 MOE models [3/N] (#42667) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-06-12 20:44:52 +00:00
Matthew BonanniandGitHub cf567cbc71 [Attention] Improve attention benchmarks: configs and profiling (#39336) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-06-12 16:24:25 -04:00
Micah WilliamsonandGitHub 39cb9bf292 [ROCm] Bump Torch to 2.11 (#45362) 
Signed-off-by: Micah Williamson <micah.williamson@amd.com>
 2026-06-12 15:22:26 -05:00
Flora FengandGitHub 6e4a547176 [Refactor] Deprecate ResponsesParser wrapper, inline parsing into ParsableContext (#45431) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2026-06-12 16:15:41 -04:00
Ryan RockGitHubgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 		aab639c705 [Core][AMD] Propagate shutdown timeout to MultiprocExecutor (#43154) 
Signed-off-by: Ryan Rock <ryan.rock@amd.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-06-12 15:13:31 -05:00
qizixiGitHubClaude Fable 5Yongye Zhu
 		efe7adb5e1 [Perf] Use native DSA indexer decode path for next_n > 2 on SM100 (#45322) 
Signed-off-by: zixi-qi <zixi@inferact.ai>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
 2026-06-12 12:54:00 -07:00
Isotr0pyandGitHub 6635279d8a [Migration] Migrate GGUF quantization support to plugin (#39612) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-06-12 12:02:21 -07:00
Jonas I. LiechtiandGitHub d6fd7ce8da [Model][Dflash] Enable Dflash support for Qwen3NextForCausalLM targets (#45319) 
Signed-off-by: Jonas I. Liechti <j-i-l@t4d.ch>
 2026-06-12 10:30:09 -07:00
Xiaohong (Sean) ChenGitHubYanan Cao
 		272c16953e [Kernel][Helion][1/N] Add Helion kernel for dynamic_per_token_scaled_fp8_quant (#33790) 
Signed-off-by: Sean Chen <seachen@redhat.com>
Co-authored-by: Yanan Cao <gmagogsfm@gmail.com>
 2026-06-12 12:50:06 -04:00
Yi ZhongandGitHub 053e7daa79 [Model] Add encoder CUDA graph support to Lfm2VL (#44930) 
Signed-off-by: vincentzed <207368749+vincentzed@users.noreply.github.com>
 2026-06-12 09:17:26 -07:00
Tahsin TunanGitHubBugen Zhaomergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		5af4aec141 [Rust Frontend] Add standalone granite4 tool parser (#45216) 
Signed-off-by: Tahsin Tunan <tahsintunan@gmail.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-06-13 00:16:36 +08:00
Sai Sridhar TarraandGitHub a30addc754 [Docs][KV Connector][NIXL] document KV Transfer stat logging and Prometheus metrics (#44055) 
Signed-off-by: Sai Sridhar <tarrasridhar1154@gmail.com>
 2026-06-12 15:39:11 +00:00
ChaunceyandGitHub 3b8fc3fe6d [Frontend] Support strict mode for tool calling with ResponsesAPI (#45396) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2026-06-12 10:59:59 -04:00
Srinivas KrovvidiGitHubsrinivas_oo7
 		9ff278b1d2 [Core][KV Connector] fix scheduler KV connector stats aggregation (#43877) 
Fixes scheduler-side KV connector stats collection so that:

1. update_connector_output() runs before scheduler-side stats are collected.
2. worker-side and scheduler-side KV connector stats are aggregated when both are present.
3. scheduler-only KV connector stats are still emitted when no worker-side stats exist.

Signed-off-by: srinivas_oo7 <sklinkedin0120@gmail.com>
Co-authored-by: srinivas_oo7 <sklinkedin0120@gmail.com>
 2026-06-12 14:51:55 +00:00
Guan-Ming (Wesley) ChiuandGitHub c7aa3d2630 [Core] Support structured outputs for beam search (#35022) 
Signed-off-by: Guan-Ming (Wesley) Chiu <guanmingchiu@gmail.com>
Signed-off-by: Guan-Ming (Wesley) Chiu <105915352+guan404ming@users.noreply.github.com>
 2026-06-12 06:56:25 -07:00
Wentao YeGitHubmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		fbc3a1907a [Bug] Migrate Reset cache for both v2 and v1 model runner (#42759) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-06-12 09:38:12 -04:00