karylab_agents/vllm
1
0
Fork 0
forked from Karylab-cklius/vllm
Code Pull Requests 2 Actions 1 Packages Activity
17,864 Commits 354 Branches 175 Tags
2a09d50034c19ef4ab8564cdb53ddf4bed29a323
Commit Graph
289 Commits
Author SHA1 Message Date
Samuel ShenandGitHub c9135db27c [Docs] Update stale LMCache examples (#45762) 
Signed-off-by: Samuel Shen <slshen@tensormesh.ai>
 2026-06-19 03:21:36 +00:00
Yifan QiaoGitHubNick Hill
 		35e4dd4a69 [KV Connector][Mooncake] Async lookup to reduce scheduler overhead (#45659) 
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
 2026-06-18 21:44:02 +00:00
AlexHuangGitHubClaudeKunshang Jimergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		8d4f54966c fix(quantization): Fix AWQ dequantize on Intel XPU and refactor AutoAWQ config (#42727) 
Signed-off-by: Alex <alex.tech.lab@outlook.com>
Signed-off-by: AlexHuang <jihuihuang@tencent.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-06-18 20:12:28 +08:00
Angelo RuoccoandGitHub ee0fd6984a docs, kv_offloading: add docs for selective offload (#45279) 
Signed-off-by: Angelo Ruocco <ang@zurich.ibm.com>
 2026-06-17 14:58:00 +03:00
Dao007foreverGitHubClaude Opus 4.8
 		44b2512767 [KV Connector][Mooncake] Add cache_prefix to namespace store keys (#45767) 
Signed-off-by: Dao Le <Dao007forever@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-16 10:24:20 -07:00
Flora FengandGitHub cd9078fe59 [Frontend] Skip structural tags for auto tool_choice without strict mode (#45600) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2026-06-15 19:55:31 +00:00
Ekagra RanjanGitHubgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>Harry MellorBenjamin Chislett
 		4ef4492e9b [V1][Spec Decode] Add Dynamic SD (#32374) 
Signed-off-by: Ekagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com>
Signed-off-by: Benjamin Chislett <chislett.ben@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com>
 2026-06-14 00:14:27 -07:00
Isotr0pyandGitHub 6635279d8a [Migration] Migrate GGUF quantization support to plugin (#39612) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-06-12 12:02:21 -07:00
Sai Sridhar TarraandGitHub a30addc754 [Docs][KV Connector][NIXL] document KV Transfer stat logging and Prometheus metrics (#44055) 
Signed-off-by: Sai Sridhar <tarrasridhar1154@gmail.com>
 2026-06-12 15:39:11 +00:00
ChaunceyGitHubcjackalmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		2043258dec [Frontend] Support strict mode for tool calling (#45003) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
Co-authored-by: cjackal <44624812+cjackal@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-06-12 07:51:48 +00:00
Fynn Schmitt-UlmsGitHubmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		ebc6ef971a Hidden states extraction improvements (#43805) 
Signed-off-by: Fynn Schmitt-Ulms <fschmitt@redhat.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-06-11 09:44:45 -04:00
LiGitHubLi ZhangClaudeChauncey
 		89c6a41001 [Bench] Add BFCL dataset for vllm bench serve tool-calling workloads (#42457) 
Signed-off-by: Li Zhang <lzhanga@amazon.com>
Co-authored-by: Li Zhang <lzhanga@amazon.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
 2026-06-09 23:59:18 -07:00
Change72andGitHub 320c52b134 [Bench] benchmark_serving_multi_turn: make non-standard conversation_id payload opt-in (#43756) 
Signed-off-by: Change72 <cguo51@asu.edu>
 2026-06-09 19:41:56 -07:00
Ronen SchafferGitHubmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		3d119f78f7 [Docs] Add KV offloading usage guide (single- and multi-tier) (#44415) 
Signed-off-by: Ronen Schaffer <ronen.schaffer@ibm.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-06-09 19:20:23 +03:00
Mohammad Miadh AngkadandGitHub fff9210b2a [CI/Docs] Remove stale disagg prefill links (#44918) 
Signed-off-by: Mohammad Miadh Angkad <176301910+mmangkad@users.noreply.github.com>
 2026-06-09 03:05:53 -07:00
Lanze LiuandGitHub 4128605ad4 [Docs] Remove broken link to deleted disaggregated_prefill.sh (#44929) 
Signed-off-by: Lanze Liu <lanzetech@gmail.com>
 2026-06-09 01:40:06 +00:00
Nicolò LucchesiandGitHub 5add018beb [Connector] Remove P2pNcclConnector (#44854) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2026-06-08 18:58:29 +08:00
Daoyuan LiandGitHub f6a708ab2b [Doc] Add Llama-3.2-3B-Instruct to batch-invariance tested models (#44435) 
Signed-off-by: Daoyuan Li <94409450+DaoyuanLi2816@users.noreply.github.com>
 2026-06-05 16:04:32 -07:00
Harry MellorandGitHub a80af24356 Speed up docs build (#44635) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-06-05 14:51:44 +00:00
Nicolò LucchesiandGitHub d98b8f371c [NixlConnector] Initiate deprecation cycle for kv_both role (#43874) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2026-06-05 11:08:17 +02:00
Fadi ArafehandGitHub 3da29aa4a5 [DOC] Add INT8 W4A8 docs and Arm's supported quantization schemes (#34894) 
Signed-off-by: Fadi Arafeh <fadi.arafeh@arm.com>
 2026-06-04 16:27:17 +00:00
Luciano MartinsGitHubLuciano Martins
 		a248b45d05 [Model] Add Gemma4 Unified (encoder-free) support (#44429) 
Signed-off-by: Luciano Martins <lucianommartins@users.noreply.github.com>
Co-authored-by: Luciano Martins <lucianommartins@users.noreply.github.com>
 2026-06-03 12:01:39 -07:00
Isotr0pyandGitHub 1fd8bd02a4 [Docs] Replace broken video url in examples (#44159) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-06-01 06:01:10 +00:00
JINO ROHITandGitHub e1814f822d minor docs: fix incorrect example path (#43830) 
Signed-off-by: JINO-ROHIT <find.jinorohit@gmail.com>
 2026-05-27 22:58:43 -07:00
Ashwin GiridharanGitHubChauncey
 		52a31ccecc [Bugfix] Map reasoning_effort to enable_thinking in chat template kwargs (#43401) 
Signed-off-by: Ashwin Giridharan <girida@amazon.com>
Signed-off-by: Chauncey <chaunceyjiang@gmail.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
 2026-05-27 05:39:49 -07:00
Simon DanielssonGitHubgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 		d565357a90 [Docs][ROCm] MoRI-IO Connector Usage Guide (#43603) 
Signed-off-by: simondanielsson <simon.danielsson99@hotmail.com>
Signed-off-by: Simon Danielsson <70206058+simondanielsson@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-05-26 21:52:30 +08:00
Dao007foreverGitHubClaude
 		0902d8e62f [KV Connector] Keep MooncakeStore full hits block-aligned (#43494) 
Signed-off-by: Dao Le <daole@inferact.ai>
Signed-off-by: Dao Le <Dao007forever@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-05-23 23:15:03 -07:00
Benjamin ChislettGitHubgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 		4e2eba28be [Perf] Optimize hidden state extraction logic (#37374) 
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
Signed-off-by: Benjamin Chislett <chislett.ben@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-05-22 18:23:08 -04:00
KebeandGitHub 19cf334207 [Feature] Support manually enabling the cumem allocator (#33648) 
Signed-off-by: Kebe <mail@kebe7jun.com>
 2026-05-20 08:58:30 -04:00
Nicolò LucchesiandGitHub 40651c0207 [Docs][PD][NIXL] Bidirectional kv-cache transfer (#43097) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2026-05-20 09:02:36 +02:00
Dao007foreverGitHubClaude
 		aed2eb355a [Docs] Fix MooncakeStoreConnector role in disaggregated example (#42994) 
Signed-off-by: Dao Le <Dao007forever@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-05-19 11:14:43 -07:00
wang.yuqiGitHubgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>Cyrus LeungHarry Mellor
 		257af77bc2 [Docs] Reorganize online serving docs. (#41907) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-05-19 14:43:18 +08:00
Blanc SwanandGitHub 4a39b4f553 [Model] Add Apertus Tool Parser (#41154) 
Signed-off-by: Blanc <swan.blanc@infomaniak.com>
 2026-05-18 11:20:04 -04:00
Jee Jee LiGitHubgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>Roger Wang
 		7d5b033782 [LoRA] Support 2D and 3D MoE LoRA adapter at the same time (#42242) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
 2026-05-18 15:22:26 +08:00
Zhewen LiGitHubZhewen LiClaude Opus 4.7
 		36e74c9ea4 [KV Connector] Support disk offloading in MooncakeStoreConnector (#42689) 
Signed-off-by: Zhewen Li <zhewenli@inferact.ai>
Co-authored-by: Zhewen Li <zhewenli@inferact.ai>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-16 13:34:15 -07:00
Cyrus LeungandGitHub 2676ab1e0b [Deprecation] Remove old locations of get_tokenizer and resolve_hf_chat_template (#35024) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-05-15 00:13:32 -07:00
Michael GoinandGitHub 8efd508204 [Quantization] Rework quantization_config to use QuantKey and allow for activation override (#41566) 2026-05-13 16:58:32 -04:00
CynicDoraandGitHub 256dbcaabf [Feature] Support custom callable proposer backend for speculative decoding (#39487) 
Signed-off-by: 524031910363 <hyzhyzsh@sjtu.edu.cn>
Signed-off-by: CynicDora <hyzhyzsh@sjtu.edu.cn>
 2026-05-13 16:53:01 +00:00
+4
Chao LeiGitHubleichao.lcivaniumaoshen524Dao007foreverTeng MaPz1116foraxeSkywalker-EPfems14jianzsbaxingpiaochongmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		ebeb09d822 [KV Transfer] Add MooncakeStoreConnector for KV cache offloading via Mooncake distributed store (#40900) 
Signed-off-by: leichao.lc <leichao.lc@antgroup.com>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: leichao.lc <leichao.lc@antgroup.com>
Co-authored-by: ivanium <yifanqiao@inferact.ai>
Co-authored-by: aoshen524 <aoshen@inferact.ai>
Co-authored-by: Dao007forever <daole@inferact.ai>
Co-authored-by: Teng Ma <sima.mt@alibaba-inc.com>
Co-authored-by: Pz1116 <zpbzpb123123@gmail.com>
Co-authored-by: foraxe <1055696449@qq.com>
Co-authored-by: Skywalker-EP <173423846@qq.com>
Co-authored-by: fems14 <1804143737@qq.com>
Co-authored-by: jianzs <zheng.shoujian@outlook.com>
Co-authored-by: baxingpiaochong <771405853@qq.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-05-12 16:09:10 -07:00
Nicolò LucchesiandGitHub 770e9bd6b3 [Nixl][PD] Lease renewal TTL KV blocks on P (#41383) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2026-05-11 09:27:30 +00:00
Abhishek GuptaGitHubAbhiOnGithub
 		27d3bac272 docs: clarify Gemma 4 assistant speculative decoding (#42180) 
Signed-off-by: AbhiOnGithub <abhiOnGithub@users.noreply.github.com>
Co-authored-by: AbhiOnGithub <abhiOnGithub@users.noreply.github.com>
 2026-05-09 20:08:44 -07:00
Simon DanielssonandGitHub f9b9bf3bbb [CI][ROCm] Ship RIXL with vllm/vllm-openai-rocm (#41634) 
Signed-off-by: simondanielsson <simon.danielsson99@hotmail.com>
 2026-05-08 07:05:17 +00:00
wang.yuqiGitHubgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 		1d694e78c9 [Examples][last/6] Resettle examples. (#41084) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-05-07 19:42:12 -07:00
wang.yuqiGitHubgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 		51c1ee9b7c [Examples] Resettle Disaggregated examples. (#40759) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-05-06 01:20:38 -07:00
Taneem IbrahimandGitHub 54dc64d5d3 [Doc] Add Qwen3-30B-A3B-Thinking-2507-FP8 to batch invariance verified models (#41513) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
 2026-05-03 08:47:55 -04:00
Dong WandGitHub 7198940b39 [Model] Add Moondream3 model support(only query and caption skills) (#32325) 
Signed-off-by: Dong Wang <dongw2019@gmail.com>
 2026-05-01 10:06:48 +08:00
Luis 🚀GitHubAndrew Sansom
 		14043dfecd feat: Enable prompt_embeds Content Part Support in vLLM Chat Completions API (#40720) 
Signed-off-by: Luis Robaina <luis@protopia.ai>
Signed-off-by: Luis Robaina 🚀 <luisfabian1545@gmail.com>
Signed-off-by: LuisRobaina <luis@protopia.ai>
Co-authored-by: Andrew Sansom <qthequartermasterman@gmail.com>
 2026-05-01 10:05:55 +08:00
ChaunceyGitHubmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		faab189554 [Feature]: IndexCache support for DSA models (#37735) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-04-29 15:15:35 -04:00
Walter Beller-MoralesandGitHub 1312f07531 [Feature] add cohere reasoning and tool parsers (#40422) 
Signed-off-by: walterbm <walter.beller.morales@gmail.com>
 2026-04-28 21:07:53 -07:00
wang.yuqiandGitHub a8208e6a81 [Examples] Resettle features examples. (#40995) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-04-28 00:33:41 -07:00