karylab_agents/vllm
1
0
Fork 0
forked from Karylab-cklius/vllm
Code Pull Requests 2 Actions 1 Packages Activity
16,919 Commits 354 Branches 175 Tags
1806d1adfc9b598bc6eb94de38a330aaad04c291
Commit Graph
263 Commits
Author SHA1 Message Date
Dao007foreverGitHubClaude
 		0902d8e62f [KV Connector] Keep MooncakeStore full hits block-aligned (#43494) 
Signed-off-by: Dao Le <daole@inferact.ai>
Signed-off-by: Dao Le <Dao007forever@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-05-23 23:15:03 -07:00
Benjamin ChislettGitHubgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 		4e2eba28be [Perf] Optimize hidden state extraction logic (#37374) 
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
Signed-off-by: Benjamin Chislett <chislett.ben@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-05-22 18:23:08 -04:00
KebeandGitHub 19cf334207 [Feature] Support manually enabling the cumem allocator (#33648) 
Signed-off-by: Kebe <mail@kebe7jun.com>
 2026-05-20 08:58:30 -04:00
Nicolò LucchesiandGitHub 40651c0207 [Docs][PD][NIXL] Bidirectional kv-cache transfer (#43097) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2026-05-20 09:02:36 +02:00
Dao007foreverGitHubClaude
 		aed2eb355a [Docs] Fix MooncakeStoreConnector role in disaggregated example (#42994) 
Signed-off-by: Dao Le <Dao007forever@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-05-19 11:14:43 -07:00
wang.yuqiGitHubgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>Cyrus LeungHarry Mellor
 		257af77bc2 [Docs] Reorganize online serving docs. (#41907) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-05-19 14:43:18 +08:00
Blanc SwanandGitHub 4a39b4f553 [Model] Add Apertus Tool Parser (#41154) 
Signed-off-by: Blanc <swan.blanc@infomaniak.com>
 2026-05-18 11:20:04 -04:00
Jee Jee LiGitHubgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>Roger Wang
 		7d5b033782 [LoRA] Support 2D and 3D MoE LoRA adapter at the same time (#42242) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
 2026-05-18 15:22:26 +08:00
Zhewen LiGitHubZhewen LiClaude Opus 4.7
 		36e74c9ea4 [KV Connector] Support disk offloading in MooncakeStoreConnector (#42689) 
Signed-off-by: Zhewen Li <zhewenli@inferact.ai>
Co-authored-by: Zhewen Li <zhewenli@inferact.ai>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-16 13:34:15 -07:00
Cyrus LeungandGitHub 2676ab1e0b [Deprecation] Remove old locations of get_tokenizer and resolve_hf_chat_template (#35024) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-05-15 00:13:32 -07:00
Michael GoinandGitHub 8efd508204 [Quantization] Rework quantization_config to use QuantKey and allow for activation override (#41566) 2026-05-13 16:58:32 -04:00
CynicDoraandGitHub 256dbcaabf [Feature] Support custom callable proposer backend for speculative decoding (#39487) 
Signed-off-by: 524031910363 <hyzhyzsh@sjtu.edu.cn>
Signed-off-by: CynicDora <hyzhyzsh@sjtu.edu.cn>
 2026-05-13 16:53:01 +00:00
+4
Chao LeiGitHubleichao.lcivaniumaoshen524Dao007foreverTeng MaPz1116foraxeSkywalker-EPfems14jianzsbaxingpiaochongmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		ebeb09d822 [KV Transfer] Add MooncakeStoreConnector for KV cache offloading via Mooncake distributed store (#40900) 
Signed-off-by: leichao.lc <leichao.lc@antgroup.com>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: leichao.lc <leichao.lc@antgroup.com>
Co-authored-by: ivanium <yifanqiao@inferact.ai>
Co-authored-by: aoshen524 <aoshen@inferact.ai>
Co-authored-by: Dao007forever <daole@inferact.ai>
Co-authored-by: Teng Ma <sima.mt@alibaba-inc.com>
Co-authored-by: Pz1116 <zpbzpb123123@gmail.com>
Co-authored-by: foraxe <1055696449@qq.com>
Co-authored-by: Skywalker-EP <173423846@qq.com>
Co-authored-by: fems14 <1804143737@qq.com>
Co-authored-by: jianzs <zheng.shoujian@outlook.com>
Co-authored-by: baxingpiaochong <771405853@qq.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-05-12 16:09:10 -07:00
Nicolò LucchesiandGitHub 770e9bd6b3 [Nixl][PD] Lease renewal TTL KV blocks on P (#41383) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2026-05-11 09:27:30 +00:00
Abhishek GuptaGitHubAbhiOnGithub
 		27d3bac272 docs: clarify Gemma 4 assistant speculative decoding (#42180) 
Signed-off-by: AbhiOnGithub <abhiOnGithub@users.noreply.github.com>
Co-authored-by: AbhiOnGithub <abhiOnGithub@users.noreply.github.com>
 2026-05-09 20:08:44 -07:00
Simon DanielssonandGitHub f9b9bf3bbb [CI][ROCm] Ship RIXL with vllm/vllm-openai-rocm (#41634) 
Signed-off-by: simondanielsson <simon.danielsson99@hotmail.com>
 2026-05-08 07:05:17 +00:00
wang.yuqiGitHubgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 		1d694e78c9 [Examples][last/6] Resettle examples. (#41084) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-05-07 19:42:12 -07:00
wang.yuqiGitHubgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 		51c1ee9b7c [Examples] Resettle Disaggregated examples. (#40759) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-05-06 01:20:38 -07:00
Taneem IbrahimandGitHub 54dc64d5d3 [Doc] Add Qwen3-30B-A3B-Thinking-2507-FP8 to batch invariance verified models (#41513) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
 2026-05-03 08:47:55 -04:00
Dong WandGitHub 7198940b39 [Model] Add Moondream3 model support(only query and caption skills) (#32325) 
Signed-off-by: Dong Wang <dongw2019@gmail.com>
 2026-05-01 10:06:48 +08:00
Luis 🚀GitHubAndrew Sansom
 		14043dfecd feat: Enable prompt_embeds Content Part Support in vLLM Chat Completions API (#40720) 
Signed-off-by: Luis Robaina <luis@protopia.ai>
Signed-off-by: Luis Robaina 🚀 <luisfabian1545@gmail.com>
Signed-off-by: LuisRobaina <luis@protopia.ai>
Co-authored-by: Andrew Sansom <qthequartermasterman@gmail.com>
 2026-05-01 10:05:55 +08:00
ChaunceyGitHubmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		faab189554 [Feature]: IndexCache support for DSA models (#37735) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-04-29 15:15:35 -04:00
Walter Beller-MoralesandGitHub 1312f07531 [Feature] add cohere reasoning and tool parsers (#40422) 
Signed-off-by: walterbm <walter.beller.morales@gmail.com>
 2026-04-28 21:07:53 -07:00
wang.yuqiandGitHub a8208e6a81 [Examples] Resettle features examples. (#40995) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-04-28 00:33:41 -07:00
wang.yuqiandGitHub 8d8062d0a7 [Examples] Resettle generate examples. (#36464) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-04-27 07:48:37 +00:00
Zhanda ZhuGitHubYubo Gao
 		5d5c776444 [Perf] FP8 FlashInfer Attn for ViT (#38065) 
Signed-off-by: Zhanda Zhu <zhandazhu@gmail.com>
Co-authored-by: Yubo Gao <ybgao-nvidia@users.noreply.github.com>
 2026-04-27 13:44:15 +08:00
labAxiaomingGitHubCyrus Leung
 		f768b4473e [Docs] Add docs for context extension using the yarn method (#37430) 
Signed-off-by: xiaoming <1259730330@qq.com>
Signed-off-by: labAxiaoming <34019940+labAxiaoming@users.noreply.github.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2026-04-24 08:26:09 -07:00
xiaoGitHubWangxiaoxiaoa
 		ecbe42e991 [Doc] Clarify supported keys for --speculative-config (#40455) 
Signed-off-by: Wangxiaoxiaoa <Wangxiaoxiaoa@users.noreply.github.com>
Co-authored-by: Wangxiaoxiaoa <Wangxiaoxiaoa@users.noreply.github.com>
 2026-04-22 04:36:17 -07:00
storyiconGitHubmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		6aa057c9d7 [Multimodal] Support custom video metadata for pre-extracted frame sequences (#40133) 
Signed-off-by: storyicon <storyicon@foxmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-04-22 15:50:04 +08:00
Yusuf MohammadGitHubmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		ec5ef0ac73 [Doc] Add Qwen3 AWQ models to documentation (#40034) 
Signed-off-by: Yusuf <yusufmohammad@live.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-04-21 09:37:41 -04:00
Vasiliy KuznetsovandGitHub 38fa87caca mxfp8 online quant move to new frontend (#40152) 
Signed-off-by: Vasiliy Kuznetsov <vasiliy@meta.com>
 2026-04-20 06:26:12 -07:00
milesialandGitHub 6d8b80802b [Docs] Fix thinking_token_budget docs (#40316) 
Signed-off-by: milesial <milesial@users.noreply.github.com>
 2026-04-20 08:09:44 +00:00
Nick CaoGitHubClaude
 		153ba7f0f3 [Refactor] Drop direct dependency on librosa (#39079) 
Signed-off-by: Nick Cao <ncao@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-04-18 06:55:38 +00:00
Vasiliy KuznetsovandGitHub 5e5afafa21 [Doc] add docs for online quant frontend (#39736) 
Signed-off-by: Vasiliy Kuznetsov <vasiliy@meta.com>
 2026-04-16 07:52:58 -07:00
Nicolò LucchesiandGitHub 3244a2ebf2 [KVConnector][NIXL] Organize NIXL connector into its own directory (#39354) 
The number of features supported by the connector has grown substantially
and the `nixl_connector.py` file has accumulated a lot of code. Creates a separate
directory and isolates connector/scheduler code in the hope of improving clarity
and maintainability.

Further refactor of components aimed at improving clarity and simplifying code
will follow soon.

Signed-off-by: NickLucche <nlucches@redhat.com>
 2026-04-12 13:10:50 +00:00
ChaunceyandGitHub ecbfbb8d61 [Feature] Add auto-detection for reasoning_config when only reasoning_parser is set (#38214) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2026-04-10 01:36:26 +00:00
ChaunceyandGitHub cbe7d18096 [Misc] Rename think_start_str/think_end_str to reasoning_start_str/reasoning_end_str (#38242) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2026-04-01 09:56:45 -07:00
Nicolò LucchesiandGitHub 7337ff7f03 [Docs] PD with Nixl compat matrix (#38628) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2026-03-31 15:01:21 +00:00
Flora FengandGitHub 3e802e8786 [Mypy] Fix adjust_request typing (#38264) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2026-03-31 04:21:18 +00:00
Cyrus LeungandGitHub ba2f0acc2d [Misc] Reorganize inputs (#35182) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-03-25 10:22:54 -07:00
Sungjae LeeGitHubChaunceyHarry Mellor
 		4731884796 [Feature] limit thinking tokens (hard limit) (#20859) 
Signed-off-by: Sungjae Lee <33976427+llsj14@users.noreply.github.com>
Signed-off-by: Sungjae Lee <sung-jae.lee@navercorp.com>
Signed-off-by: Chauncey <chaunceyjiang@gmail.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-03-24 09:53:07 -07:00
Yan MaandGitHub d3fe857135 update doc for online fp8 quantization (#37851) 
Signed-off-by: Yan Ma <yan.ma@intel.com>
 2026-03-23 05:19:03 +00:00
+3
Ifta khairul Alam AdilGitHubJartXNetanel HaberWentao YePhilip OttesenWoosuk KwonMichael GoinGiancarlo DelfinAndy LoThillai Chithambaramsihao_liHarry Mellor
 		104605cbf2 Remove deprecated reasoning_content message field(part-2) (#37480) 
Signed-off-by: JartX <sagformas@epdcenter.es>
Signed-off-by: Ifta Khairul Alam Adil <ikaadil007@gmail.com>
Signed-off-by: Netanel Haber <58652339+netanel-haber@users.noreply.github.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Philip Ottesen <phiott256@gmail.com>
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
Signed-off-by: Andy Lo <andy@mistral.ai>
Signed-off-by: Thillai Chithambaram <thillaichithambaram.a@gmail.com>
Signed-off-by: sihao.li <sihao.li@intel.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: JartX <sagformas@epdcenter.es>
Co-authored-by: Netanel Haber <58652339+netanel-haber@users.noreply.github.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: Philip Ottesen <phiott256@gmail.com>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Giancarlo Delfin <32987265+TheEpicDolphin@users.noreply.github.com>
Co-authored-by: Andy Lo <andy@mistral.ai>
Co-authored-by: Thillai Chithambaram <79466435+thillai-c@users.noreply.github.com>
Co-authored-by: sihao_li <165983188+1643661061leo@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-03-19 15:20:08 +00:00
wang.yuqiGitHubCyrus LeungHarry Mellor
 		f9e2a38386 [Docs] Reorganize pooling docs. (#35592) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-03-19 11:25:47 +00:00
BhoomitGitHubHarry Mellor
 		3717a4dd47 [Misc][LoRA] Add --lora-target-modules to restrict LoRA to specific modules (#34984) 
Signed-off-by: Bhoomit Vasani <bhoomit.2010@gmail.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-03-17 14:36:41 +00:00
Yuchen FamaandGitHub 31a458c091 [Doc] Clarify schema enforcement behavior for tool_choice modes (#37064) 
Signed-off-by: yfama <yuchengu@gmail.com>
 2026-03-16 22:27:42 +00:00
Max de BayserandGitHub 9f9ecff4cd Add simple granite4 tool parser (#36827) 
Signed-off-by: Max de Bayser <maxdebayser@gmail.com>
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
 2026-03-16 10:49:09 -07:00
sfeiqiangGitHubphaedonsun
 		8cb24d3aed [KV Connector] Support using FlexKV as KV Cache Offloading option. (#34328) 
Signed-off-by: phaedonsun <phaedonsun@tencent.com>
Co-authored-by: phaedonsun <phaedonsun@tencent.com>
 2026-03-12 00:46:20 -07:00
Harry MellorandGitHub a0f44bb616 Allow markdownlint to run locally (#36398) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-03-08 20:05:24 -07:00
Xiang ShiandGitHub e68de8adc0 docs: fix wrong cc in int8.md (#36209) 
Signed-off-by: Xiang Shi <realkevin@tutanota.com>
 2026-03-06 06:01:02 +00:00