karylab_agents/vllm
1
0
Fork 0
forked from Karylab-cklius/vllm
Code Pull Requests 2 Actions 1 Packages Activity
17,864 Commits 354 Branches 175 Tags
2a09d50034c19ef4ab8564cdb53ddf4bed29a323
Commit Graph
893 Commits
Author SHA1 Message Date
Samuel ShenandGitHub c9135db27c [Docs] Update stale LMCache examples (#45762) 
Signed-off-by: Samuel Shen <slshen@tensormesh.ai>
 2026-06-19 03:21:36 +00:00
Luciano MartinsGitHubLuciano Martins
 		58b2e89642 [Bugfix][Gemma4] Render reasoning on assistant turns without tool_calls (#45867) 
Signed-off-by: Luciano Martins <lucianommartins@users.noreply.github.com>
Co-authored-by: Luciano Martins <lucianommartins@users.noreply.github.com>
 2026-06-17 20:44:15 +00:00
Oğuzhan KIRandGitHub fa85ead2f3 [MM][Perf][CG] Support ViT full CUDA graph for Kimi-VL (#41992) 
Signed-off-by: oguz <oguzhankir17@gmail.com>
 2026-06-17 12:14:01 +00:00
Shanshan ShenGitHubRoger WangIsotr0py
 		ad32608e24 [MM][Perf][CG] Support dual-path ViT full CUDA graph for DeepSeek-OCR (#43586) 
Signed-off-by: shen-shanshan <467638484@qq.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-06-16 04:35:20 -07:00
Luciano MartinsGitHubLuciano Martins
 		6607a80dab [Bugfix][Gemma4] Fix offline parser truncation, adjust_request token leak, and chat template sync (#45553) 
Signed-off-by: Luciano Martins <lucianommartins@users.noreply.github.com>
Co-authored-by: Luciano Martins <lucianommartins@users.noreply.github.com>
 2026-06-16 04:31:53 +00:00
snadampalGitHubNicolò Lucchesi
 		88ed636218 [KV Connector]: Support KV push from Prefill to Decode node using Nixl KV Connector (#35264) 
Signed-off-by: Sunita Nadampalli <nadampal@amazon.com>
Signed-off-by: NickLucche <nlucches@redhat.com>
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com>
 2026-06-12 10:38:41 +00:00
allgatherGitHubIsotr0py
 		39dee1114a [MM][Perf][CG] Support ViT full cudagraphs for mllama4 (#40660) 
Signed-off-by: allgather <all2allops@gmail.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-06-11 22:17:55 -07:00
Fynn Schmitt-UlmsGitHubmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		ebc6ef971a Hidden states extraction improvements (#43805) 
Signed-off-by: Fynn Schmitt-Ulms <fschmitt@redhat.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-06-11 09:44:45 -04:00
Nathan PriceGitHubCursorCyrus Leungmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		12f3f19c19 feat(qwen3-asr): support prompt parameter in v1/audio/transcriptions (#35415) 
Signed-off-by: Nathan Price <nathan@abridge.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-06-10 19:54:59 +00:00
Peter PanGitHubHarry MellorKyle SayersFlora Feng
 		3d300aecb1 [Doc] Switch K8S examples to default MP mode (#39400) 
Signed-off-by: Peter Pan <Peter.Pan@daocloud.io>
Signed-off-by: Peter Pan <peter.pan@daocloud.io>
Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Kyle Sayers <kylesayrs@gmail.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-06-10 18:17:11 +00:00
Harry MellorandGitHub 3cc9fecd58 Deprecated 1st generation Qwen and QwenVL models (#45131) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-06-10 14:55:33 +00:00
Change72andGitHub 320c52b134 [Bench] benchmark_serving_multi_turn: make non-standard conversation_id payload opt-in (#43756) 
Signed-off-by: Change72 <cguo51@asu.edu>
 2026-06-09 19:41:56 -07:00
SageandGitHub 5b3807e862 [KV Events] Switch event structs from array to map encoding (#42892) 
Signed-off-by: Sage Ahrac <sagiahrak@gmail.com>
 2026-06-09 11:39:52 +00:00
GuorongYeGitHubgrYe99Isotr0py
 		9f153aa781 [MM][Perf][CG] Support ViT full CUDA graph for glm4_1v image and video inference (#40576) 
Signed-off-by: grYe99 <guorongye99@gmail.com>
Co-authored-by: grYe99 <guorongye99@gmail.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-06-09 11:13:56 +08:00
Nicolò LucchesiandGitHub 5add018beb [Connector] Remove P2pNcclConnector (#44854) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2026-06-08 18:58:29 +08:00
Oğuzhan KIRGitHubIsotr0py
 		f25952e59b [MM][Perf][CG] Support ViT full CUDA graph for InternVL (#41759) 
Signed-off-by: oguz <oguzhankir17@gmail.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-06-04 10:24:25 +08:00
Siddharth BedekarGitHubOpenAI Codex
 		266b9d9c64 [Frontend][Core] Add sparse NCCL weight transfer support for in-place updates (#40096) 
Signed-off-by: Siddharth Bedekar <bedeksid@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-06-01 15:37:30 -04:00
Isotr0pyandGitHub 1fd8bd02a4 [Docs] Replace broken video url in examples (#44159) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-06-01 06:01:10 +00:00
Harshal JanjaniGitHubBen Browning
 		69c9f19957 fix(frontend): Add multimodal placeholders to Gemma4 tool message template (#41459) 
Signed-off-by: Harshal Janjani <harshaljanjani@gmail.com>
Co-authored-by: Ben Browning <bbrownin@redhat.com>
 2026-05-28 14:48:12 -07:00
Andreas KaratzasandGitHub 76ea1d5d2f [ROCm][CI] Stabilize Granite tool-use and test URL construction (#43017) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-05-23 12:21:11 +08:00
Benjamin ChislettGitHubgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 		4e2eba28be [Perf] Optimize hidden state extraction logic (#37374) 
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
Signed-off-by: Benjamin Chislett <chislett.ben@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-05-22 18:23:08 -04:00
Aaron HaoGitHubmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		f34623bf3c [bug] AsyncScheduler drops first post-resume token after pause_generation + clear_cache (#42117) 
Signed-off-by: hao-aaron <ahao@anyscale.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-05-19 01:06:21 -07:00
Blanc SwanandGitHub 4a39b4f553 [Model] Add Apertus Tool Parser (#41154) 
Signed-off-by: Blanc <swan.blanc@infomaniak.com>
 2026-05-18 11:20:04 -04:00
SoyaazzandGitHub 990f49bdcb [MM][CG] Enable encoder Cudagraph for Step3VL (#42224) 
Signed-off-by: JisoLya <523420504@qq.com>
Signed-off-by: Soyaazz <523420504@qq.com>
 2026-05-17 20:19:13 -07:00
Aaron HaoandGitHub e0a45f1455 [Feat][RL] IPC weight sync optimizations: multigpu support and chunked packed tensors (#37476) 
Signed-off-by: ahao-anyscale <ahao@anyscale.com>
Signed-off-by: hao-aaron <ahao@anyscale.com>
 2026-05-15 22:53:06 +08:00
Krish GuptaandGitHub 70c00163ff [Feature] Add instruction support for score/rerank chat templates (#42412) 
Signed-off-by: KrxGu <krishom70@gmail.com>
 2026-05-14 09:41:22 +08:00
John CalderonGitHubRoger Wang
 		b3c69595a6 [MM][CG] Support ViT CG for Qwen2-VL (#41736) 
Signed-off-by: John Calderon <jcalderon@nvidia.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
 2026-05-14 01:52:35 +08:00
Shanshan ShenandGitHub 92def124bc [MM][Perf][CG] Support ViT full CUDA graph for Qwen3.5 (#42151) 
Signed-off-by: shen-shanshan <467638484@qq.com>
 2026-05-13 16:00:32 +08:00
yzong-rhandGitHub b1687527b8 [Bugfix] Gemma 4 chat template crash with missing tool name and tool id (#42188) 
Signed-off-by: Yifan <yzong@redhat.com>
 2026-05-11 03:07:45 +00:00
Wentao YeandGitHub ea0e501bb1 [KV Connector] Remove compat support for pre-v0.12.0 constructor signatures without KVCacheConfig (#39832) 
The v0.12.0 release contained initial support for HMA in KV Connectors. As part
of these changes, a KVCacheConfig argument was added to KV connector
constructors. Backwards compatibility support for out-of-tree connectors was
included in this change, with a very prominent warning. See #25712 and #27887.

Since the warning has been around for over 5 months, we can safely remove
the support of it.

Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-05-09 23:39:46 +00:00
Sumanth R HegdeandGitHub e3b65a5ba0 [feat] Add explicit /start_weight_update and /finish_weight_update APIs for weight transfer (#39212) 2026-05-08 18:03:33 -07:00
Ethan FengandGitHub 4140faa4a5 [Docs] Fix OpenAI batch model argument examples (#42066) 
Signed-off-by: Ethan Feng <ethan.fengch@gmail.com>
 2026-05-08 14:02:46 +00:00
haosdentandGitHub 52458b60a8 [CI][Examples][RLHF] Disable async scheduling in rlhf_async_new_apis (#42042) 
Signed-off-by: haosdent <haosdent@gmail.com>
 2026-05-08 04:58:48 -07:00
wang.yuqiGitHubgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 		1d694e78c9 [Examples][last/6] Resettle examples. (#41084) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-05-07 19:42:12 -07:00
tejandGitHub 8a4888be21 [ROCm] Profiler api support for ROCm MORI toy proxy server in PD Disaggregation (#40264) 
Signed-off-by: Tej Kiran <kiran.tej@amd.com>
 2026-05-07 16:58:38 +08:00
wang.yuqiGitHubgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 		51c1ee9b7c [Examples] Resettle Disaggregated examples. (#40759) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-05-06 01:20:38 -07:00
John CalderonandGitHub 964a4bc2a5 [MM][CG] Support ViT CG for Qwen2.5-VL (#40830) 
Signed-off-by: John Calderon <jcalderon@nvidia.com>
 2026-05-02 11:10:14 +08:00
FredericOdermattandGitHub c408fdd663 [Fix] Sync gemma4 chat template from hf (#39570) 
Signed-off-by: Frederic Odermatt <frederic.odermatt@44ai.ch>
 2026-05-02 03:06:54 +00:00
Luis 🚀GitHubAndrew Sansom
 		14043dfecd feat: Enable prompt_embeds Content Part Support in vLLM Chat Completions API (#40720) 
Signed-off-by: Luis Robaina <luis@protopia.ai>
Signed-off-by: Luis Robaina 🚀 <luisfabian1545@gmail.com>
Signed-off-by: LuisRobaina <luis@protopia.ai>
Co-authored-by: Andrew Sansom <qthequartermasterman@gmail.com>
 2026-05-01 10:05:55 +08:00
snadampalandGitHub 3179e53135 [P/D] Prefill compute optimizations with bi-directional KV cache transfers between P and D nodes (#32553) 
Signed-off-by: Sunita Nadampalli <nadampal@amazon.com>
 2026-04-30 10:14:20 +00:00
wang.yuqiandGitHub a8208e6a81 [Examples] Resettle features examples. (#40995) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-04-28 00:33:41 -07:00
Isotr0pyandGitHub c2e88a281c [Bugfix] Fix broken example opeanai client (#41088) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-04-28 04:43:04 +00:00
wang.yuqiandGitHub 8d8062d0a7 [Examples] Resettle generate examples. (#36464) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-04-27 07:48:37 +00:00
wang.yuqiGitHubgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>Cyrus Leung
 		9744b699ba [Deprecate] Deprecate LLM.reward offline api, use LLM.encode instead. (#40688) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2026-04-24 05:37:50 +00:00
Srreyansh SethiGitHubMatthew Bonannimergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		b7a2605020 [Bugfix] Make Attention Backend Auto-Selection Batch-Invariance-Aware (#40193) 
Signed-off-by: Srreyansh Sethi <srreyansh.sethi@gmail.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-04-23 14:57:03 +00:00
Shanshan ShenandGitHub fe57be7809 [MM][CG] Support --enable-vit-cuda-graph option for VLM examples (#40580) 
Signed-off-by: shen-shanshan <467638484@qq.com>
 2026-04-22 22:46:14 -07:00
Simon DanielssonGitHubMatvei Pashkovskii
 		ac58e2a170 [Fix][MoRI] Align MoRI-IO message format with P2pNcclConnector and vllm-router (#39565) 
Signed-off-by: simondanielsson <simon.danielsson99@hotmail.com>
Co-authored-by: Matvei Pashkovskii <mpashkov@amd.com>
 2026-04-23 08:06:31 +09:00
AllenDouGitHubzixiao
 		9047288b68 support hotwords for FunASR model (#39674) 
Signed-off-by: zixiao <shunli.dsl@alibaba-inc.com>
Co-authored-by: zixiao <shunli.dsl@alibaba-inc.com>
 2026-04-22 02:25:06 -07:00
rasmithandGitHub cefa5281a7 [ROCm][P/D][MORI][BugFix] Ensure correct api is used when making requests to prefill / decode nodes (#39835) 
Signed-off-by: Randall Smith <Randall.Smith@amd.com>
 2026-04-22 09:48:25 +09:00
artem-spectorGitHubartemspectorClaude Opus 4.6
 		d249a9e90e Add Granite 4.1 Vision as built-in multimodal model (#40282) 
Signed-off-by: Artem Spector <artems@il.ibm.com>
Signed-off-by: artemspector <artems@il.ibm.com>
Co-authored-by: artemspector <artems@il.ibm.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-04-21 05:43:39 -07:00