karylab_agents/vllm
1
0
Fork 0
forked from Karylab-cklius/vllm
Code Pull Requests 2 Actions 1 Packages Activity
16,919 Commits 354 Branches 175 Tags
1806d1adfc9b598bc6eb94de38a330aaad04c291
Commit Graph
2293 Commits
Author SHA1 Message Date
Dao007foreverGitHubClaude
 		0902d8e62f [KV Connector] Keep MooncakeStore full hits block-aligned (#43494) 
Signed-off-by: Dao Le <daole@inferact.ai>
Signed-off-by: Dao Le <Dao007forever@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-05-23 23:15:03 -07:00
HolegotsandGitHub 8737e4a857 [Docs] Fix stale version number in token_classify.md (#43489) 
Signed-off-by: holegots <ikun3.1415927@gmail.com>
 2026-05-23 10:42:20 -07:00
HolegotsandGitHub 7c2ff1f819 [Docs] Fix stale version number in token_embed.md (#43488) 
Signed-off-by: holegots <ikun3.1415927@gmail.com>
 2026-05-23 10:06:56 -07:00
Duncan MossandGitHub 552bbe6f4e [Attention] Add head_dim=512 support for FlashInfer trtllm attention backend (#38822) 2026-05-22 20:27:35 -04:00
Benjamin ChislettGitHubgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 		4e2eba28be [Perf] Optimize hidden state extraction logic (#37374) 
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
Signed-off-by: Benjamin Chislett <chislett.ben@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-05-22 18:23:08 -04:00
Yongye ZhuandGitHub 843715739b [Refactor] Extract DeepSeek V4 sparse MLA impl into model folder (#43149) 2026-05-22 10:06:31 -07:00
wang.yuqiGitHubgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 		2380bfc210 [Docs] Note image preprocessing difference between qwen_vl_utils and vllm. (#43393) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-05-22 01:43:14 -07:00
Isotr0pyandGitHub ba369b7eb5 [CI] Fix dockerfile dependency graph failure for pre-commit (#43378) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-05-22 10:26:05 +08:00
Nick HillandGitHub 0f66623b0d [Frontend] Rework fastokens integration (#43168) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-05-21 15:36:58 -07:00
Nick HillGitHubBugen Zhao
 		f2ace1d57d [Frontend][RFC] Rust front-end integration (#40848) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
 2026-05-21 12:24:48 +08:00
Douglas LehrandGitHub 452baa860b Add dllehr-amd to CODEOWNERS and committers list (#42772) 
Signed-off-by: Douglas Lehr <Doug.Lehr@amd.com>
 2026-05-20 16:10:44 -05:00
KebeandGitHub 19cf334207 [Feature] Support manually enabling the cumem allocator (#33648) 
Signed-off-by: Kebe <mail@kebe7jun.com>
 2026-05-20 08:58:30 -04:00
Ray WangGitHubRui Wang
 		87e31455b0 [Doc] Sync CLI guide with actual help modes and launch subcommand (#40326) 
Signed-off-by: Rui Wang <raygorous@gmail.com>
Co-authored-by: Rui Wang <raygorous@gmail.com>
 2026-05-20 02:32:03 -07:00
Nicolò LucchesiandGitHub 40651c0207 [Docs][PD][NIXL] Bidirectional kv-cache transfer (#43097) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2026-05-20 09:02:36 +02:00
Nicolò LucchesiandGitHub 7e4bc2cecb [Docs][PD][NIXL] Lease extension mechanism for blocks on P (#43099) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2026-05-20 08:58:25 +02:00
Terrence ZhaoandGitHub 5774aaed0c [Cohere] Enable Cohere MoE (#43143) 
Signed-off-by: Terrencezzj <terrence@cohere.ai>
 2026-05-19 19:32:06 -07:00
Fadi ArafehandGitHub be16785998 [CPU][DOC] Fix installation commands for Arm CPUs (#43115) 
Signed-off-by: Fadi Arafeh <fadi.arafeh@arm.com>
 2026-05-19 23:31:15 +00:00
Dao007foreverGitHubClaude
 		aed2eb355a [Docs] Fix MooncakeStoreConnector role in disaggregated example (#42994) 
Signed-off-by: Dao Le <Dao007forever@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-05-19 11:14:43 -07:00
Wang YiwenandGitHub 1c6158083a [Model] Openvla support (#42654) 
Signed-off-by: Wang Yiwen <121547057+yiwen101@users.noreply.github.com>
 2026-05-19 08:17:42 -07:00
wang.yuqiGitHubgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>Cyrus LeungHarry Mellor
 		257af77bc2 [Docs] Reorganize online serving docs. (#41907) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-05-19 14:43:18 +08:00
Gracie Guo (UX)GitHubGracie Guowang.yuqi
 		9fd8487d2f [Docs] Add SVG images for pooling models. (#42626) 
Signed-off-by: Gracie Guo <gracieguo@Gracies-MacBook-Pro.local>
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Co-authored-by: Gracie Guo <gracieguo@Gracies-MacBook-Pro.local>
Co-authored-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-05-18 22:50:38 -07:00
Mohammad Miadh AngkadandGitHub da03e549b3 [UX] Add a persistent cache for FlashInfer autotuning (#42537) 
Signed-off-by: Mohammad Miadh Angkad <176301910+mmangkad@users.noreply.github.com>
 2026-05-18 20:25:37 -07:00
Blanc SwanandGitHub 4a39b4f553 [Model] Add Apertus Tool Parser (#41154) 
Signed-off-by: Blanc <swan.blanc@infomaniak.com>
 2026-05-18 11:20:04 -04:00
Jee Jee LiGitHubgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>Roger Wang
 		7d5b033782 [LoRA] Support 2D and 3D MoE LoRA adapter at the same time (#42242) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
 2026-05-18 15:22:26 +08:00
Harry MellorandGitHub c1f7854342 Improve logging when docs build is skipped (#42929) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-05-18 06:33:32 +00:00
SoyaazzandGitHub 990f49bdcb [MM][CG] Enable encoder Cudagraph for Step3VL (#42224) 
Signed-off-by: JisoLya <523420504@qq.com>
Signed-off-by: Soyaazz <523420504@qq.com>
 2026-05-17 20:19:13 -07:00
Zhewen LiGitHubZhewen LiClaude Opus 4.7
 		36e74c9ea4 [KV Connector] Support disk offloading in MooncakeStoreConnector (#42689) 
Signed-off-by: Zhewen Li <zhewenli@inferact.ai>
Co-authored-by: Zhewen Li <zhewenli@inferact.ai>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-16 13:34:15 -07:00
DustHunterGitHubQwen-Coderwang.yuqi
 		39c67d714e fix: add API key authorization to /v2 endpoints (#42594) 
Signed-off-by: DustHunter <dusthunter@126.com>
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
Co-authored-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-05-16 01:29:27 +00:00
Sergei SkvortsovGitHubMichael Goin
 		9a7a273dfe Add HumanEval and GSM8K benchmarks to datasets (#42648) 
Signed-off-by: southfreebird <yvorott@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2026-05-15 13:01:21 -07:00
Aaron HaoandGitHub e0a45f1455 [Feat][RL] IPC weight sync optimizations: multigpu support and chunked packed tensors (#37476) 
Signed-off-by: ahao-anyscale <ahao@anyscale.com>
Signed-off-by: hao-aaron <ahao@anyscale.com>
 2026-05-15 22:53:06 +08:00
wang.yuqiandGitHub 75fd68c7a5 [Entrypoints] Split the pooling offline API into PoolingOfflineMixin. (#42267) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-05-15 08:05:57 +00:00
Cyrus LeungandGitHub 2676ab1e0b [Deprecation] Remove old locations of get_tokenizer and resolve_hf_chat_template (#35024) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-05-15 00:13:32 -07:00
Louie TsaiandGitHub e30f39c4f1 Update Intel Xeon model list and vLLM Benchmark Suite BKMs (#42607) 
Signed-off-by: louie-tsai <louie.tsai@intel.com>
 2026-05-15 05:14:03 +00:00
Isotr0pyGitHubzxy
 		faa4b76afa [Model] Support InternS2 Preview (#42705) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: zxy <46674730+CUHKSZzxy@users.noreply.github.com>
 2026-05-14 21:30:26 -07:00
aoshen02andGitHub 8c79ad6580 Revert "[Core] Replace routing replay with device cache and async D2H pipeline" (#39917) (#42434) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
 2026-05-13 23:49:01 -07:00
Yongye ZhuGitHubRoger Wang
 		0d2732dd91 [MLA Attention Backend] Add TOKENSPEED_MLA backend for DSR1/Kimi K25 prefill + decode on Blackwell (#41778) 
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Signed-off-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Roger Wang <hey@rogerw.io>
 2026-05-13 23:48:02 -07:00
Michael GoinandGitHub 8efd508204 [Quantization] Rework quantization_config to use QuantKey and allow for activation override (#41566) 2026-05-13 16:58:32 -04:00
John CalderonGitHubRoger Wang
 		b3c69595a6 [MM][CG] Support ViT CG for Qwen2-VL (#41736) 
Signed-off-by: John Calderon <jcalderon@nvidia.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
 2026-05-14 01:52:35 +08:00
CynicDoraandGitHub 256dbcaabf [Feature] Support custom callable proposer backend for speculative decoding (#39487) 
Signed-off-by: 524031910363 <hyzhyzsh@sjtu.edu.cn>
Signed-off-by: CynicDora <hyzhyzsh@sjtu.edu.cn>
 2026-05-13 16:53:01 +00:00
Shanshan ShenandGitHub 92def124bc [MM][Perf][CG] Support ViT full CUDA graph for Qwen3.5 (#42151) 
Signed-off-by: shen-shanshan <467638484@qq.com>
 2026-05-13 16:00:32 +08:00
+4
Chao LeiGitHubleichao.lcivaniumaoshen524Dao007foreverTeng MaPz1116foraxeSkywalker-EPfems14jianzsbaxingpiaochongmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		ebeb09d822 [KV Transfer] Add MooncakeStoreConnector for KV cache offloading via Mooncake distributed store (#40900) 
Signed-off-by: leichao.lc <leichao.lc@antgroup.com>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: leichao.lc <leichao.lc@antgroup.com>
Co-authored-by: ivanium <yifanqiao@inferact.ai>
Co-authored-by: aoshen524 <aoshen@inferact.ai>
Co-authored-by: Dao007forever <daole@inferact.ai>
Co-authored-by: Teng Ma <sima.mt@alibaba-inc.com>
Co-authored-by: Pz1116 <zpbzpb123123@gmail.com>
Co-authored-by: foraxe <1055696449@qq.com>
Co-authored-by: Skywalker-EP <173423846@qq.com>
Co-authored-by: fems14 <1804143737@qq.com>
Co-authored-by: jianzs <zheng.shoujian@outlook.com>
Co-authored-by: baxingpiaochong <771405853@qq.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-05-12 16:09:10 -07:00
Juan Pérez de AlgabaGitHubmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		5a6a9fc6f6 [docs] Added one new contact to the Vulnerability Management team (#42145) 
Signed-off-by: jperezde <jperezde@redhat.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-05-12 10:59:59 -04:00
bnellnmandGitHub 6427603ae8 [MoE Refactor] Move remaining experts classes to experts directory (#42334) 
Signed-off-by: Bill Nell <bnell@redhat.com>
 2026-05-12 09:19:46 -04:00
Florian WoernerGitHubCyrus Leung
 		997132911e [Doc] Fix typo in llm-d documentation link (#42397) 
Signed-off-by: Florian Woerner <florian.woerner@onmyown.io>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2026-05-12 04:26:46 -07:00
Yasmin MoslemGitHubgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 		28ee78af54 Implement custom dataset class for ASR benchmarking (#41576) 
Signed-off-by: Yasmin Moslem <48152713+ymoslem@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-05-12 12:17:58 +08:00
Alexa GriffithGitHubClaude Sonnet 4.5
 		cf0d279142 [Docs] Add Apple Silicon documentation for vLLM-Metal GPU support (#41987) 
Signed-off-by: alexagriffith <agriffith96@gmail.com>
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
 2026-05-11 11:34:25 -07:00
Vinay R DamodaranGitHubRussell Bryant
 		5497ffbf7c Add documentation about vLLM FIPS compliance (#42190) 
Signed-off-by: Vinay Damodaran <vrdn@hey.com>
Signed-off-by: Vinay R Damodaran <vrdn@hey.com>
Co-authored-by: Russell Bryant <russell.bryant@gmail.com>
 2026-05-11 18:17:02 +00:00
Nicolò LucchesiandGitHub 770e9bd6b3 [Nixl][PD] Lease renewal TTL KV blocks on P (#41383) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2026-05-11 09:27:30 +00:00
Haoqing WangandGitHub 5cba6839e6 Document MolmoWeb hf_overrides (#42163) 
Signed-off-by: Haoqi Wang <78337154+hqhq1025@users.noreply.github.com>
 2026-05-10 23:58:22 -07:00
wangxiyuanandGitHub 5536fc0c01 [Misc] Replace mamba_type string literals with MambaAttentionBackendEnum (#41188) 
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
 2026-05-11 03:59:36 +00:00