0902d8e62f
[KV Connector] Keep MooncakeStore full hits block-aligned ( #43494 )
...
Signed-off-by: Dao Le <daole@inferact.ai >
Signed-off-by: Dao Le <Dao007forever@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-05-23 23:15:03 -07:00
Holegots and GitHub
8737e4a857
[Docs] Fix stale version number in token_classify.md ( #43489 )
...
Signed-off-by: holegots <ikun3.1415927@gmail.com >
2026-05-23 10:42:20 -07:00
Holegots and GitHub
7c2ff1f819
[Docs] Fix stale version number in token_embed.md ( #43488 )
...
Signed-off-by: holegots <ikun3.1415927@gmail.com >
2026-05-23 10:06:56 -07:00
Duncan Moss and GitHub
552bbe6f4e
[Attention] Add head_dim=512 support for FlashInfer trtllm attention backend ( #38822 )
2026-05-22 20:27:35 -04:00
4e2eba28be
[Perf] Optimize hidden state extraction logic ( #37374 )
...
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com >
Signed-off-by: Benjamin Chislett <chislett.ben@gmail.com >
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2026-05-22 18:23:08 -04:00
Yongye Zhu and GitHub
843715739b
[Refactor] Extract DeepSeek V4 sparse MLA impl into model folder ( #43149 )
2026-05-22 10:06:31 -07:00
2380bfc210
[Docs] Note image preprocessing difference between qwen_vl_utils and vllm. ( #43393 )
...
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
Signed-off-by: wang.yuqi <noooop@126.com >
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2026-05-22 01:43:14 -07:00
Isotr0py and GitHub
ba369b7eb5
[CI] Fix dockerfile dependency graph failure for pre-commit ( #43378 )
...
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn >
2026-05-22 10:26:05 +08:00
Nick Hill and GitHub
0f66623b0d
[Frontend] Rework fastokens integration ( #43168 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
2026-05-21 15:36:58 -07:00
f2ace1d57d
[Frontend][RFC] Rust front-end integration ( #40848 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
2026-05-21 12:24:48 +08:00
Douglas Lehr and GitHub
452baa860b
Add dllehr-amd to CODEOWNERS and committers list ( #42772 )
...
Signed-off-by: Douglas Lehr <Doug.Lehr@amd.com >
2026-05-20 16:10:44 -05:00
Kebe and GitHub
19cf334207
[Feature] Support manually enabling the cumem allocator ( #33648 )
...
Signed-off-by: Kebe <mail@kebe7jun.com >
2026-05-20 08:58:30 -04:00
87e31455b0
[Doc] Sync CLI guide with actual help modes and launch subcommand ( #40326 )
...
Signed-off-by: Rui Wang <raygorous@gmail.com >
Co-authored-by: Rui Wang <raygorous@gmail.com >
2026-05-20 02:32:03 -07:00
Nicolò Lucchesi and GitHub
40651c0207
[Docs][PD][NIXL] Bidirectional kv-cache transfer ( #43097 )
...
Signed-off-by: NickLucche <nlucches@redhat.com >
2026-05-20 09:02:36 +02:00
Nicolò Lucchesi and GitHub
7e4bc2cecb
[Docs][PD][NIXL] Lease extension mechanism for blocks on P ( #43099 )
...
Signed-off-by: NickLucche <nlucches@redhat.com >
2026-05-20 08:58:25 +02:00
Terrence Zhao and GitHub
5774aaed0c
[Cohere] Enable Cohere MoE ( #43143 )
...
Signed-off-by: Terrencezzj <terrence@cohere.ai >
2026-05-19 19:32:06 -07:00
Fadi Arafeh and GitHub
be16785998
[CPU][DOC] Fix installation commands for Arm CPUs ( #43115 )
...
Signed-off-by: Fadi Arafeh <fadi.arafeh@arm.com >
2026-05-19 23:31:15 +00:00
aed2eb355a
[Docs] Fix MooncakeStoreConnector role in disaggregated example ( #42994 )
...
Signed-off-by: Dao Le <Dao007forever@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-05-19 11:14:43 -07:00
Wang Yiwen and GitHub
1c6158083a
[Model] Openvla support ( #42654 )
...
Signed-off-by: Wang Yiwen <121547057+yiwen101@users.noreply.github.com >
2026-05-19 08:17:42 -07:00
257af77bc2
[Docs] Reorganize online serving docs. ( #41907 )
...
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
Signed-off-by: wang.yuqi <noooop@126.com >
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-05-19 14:43:18 +08:00
9fd8487d2f
[Docs] Add SVG images for pooling models. ( #42626 )
...
Signed-off-by: Gracie Guo <gracieguo@Gracies-MacBook-Pro.local >
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
Co-authored-by: Gracie Guo <gracieguo@Gracies-MacBook-Pro.local >
Co-authored-by: wang.yuqi <yuqi.wang@daocloud.io >
2026-05-18 22:50:38 -07:00
Mohammad Miadh Angkad and GitHub
da03e549b3
[UX] Add a persistent cache for FlashInfer autotuning ( #42537 )
...
Signed-off-by: Mohammad Miadh Angkad <176301910+mmangkad@users.noreply.github.com >
2026-05-18 20:25:37 -07:00
Blanc Swan and GitHub
4a39b4f553
[Model] Add Apertus Tool Parser ( #41154 )
...
Signed-off-by: Blanc <swan.blanc@infomaniak.com >
2026-05-18 11:20:04 -04:00
7d5b033782
[LoRA] Support 2D and 3D MoE LoRA adapter at the same time ( #42242 )
...
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com >
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai >
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Roger Wang <hey@rogerw.io >
2026-05-18 15:22:26 +08:00
Harry Mellor and GitHub
c1f7854342
Improve logging when docs build is skipped ( #42929 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-05-18 06:33:32 +00:00
Soyaazz and GitHub
990f49bdcb
[MM][CG] Enable encoder Cudagraph for Step3VL ( #42224 )
...
Signed-off-by: JisoLya <523420504@qq.com >
Signed-off-by: Soyaazz <523420504@qq.com >
2026-05-17 20:19:13 -07:00
36e74c9ea4
[KV Connector] Support disk offloading in MooncakeStoreConnector ( #42689 )
...
Signed-off-by: Zhewen Li <zhewenli@inferact.ai >
Co-authored-by: Zhewen Li <zhewenli@inferact.ai >
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com >
2026-05-16 13:34:15 -07:00
39c67d714e
fix: add API key authorization to /v2 endpoints ( #42594 )
...
Signed-off-by: DustHunter <dusthunter@126.com >
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com >
Co-authored-by: wang.yuqi <yuqi.wang@daocloud.io >
2026-05-16 01:29:27 +00:00
9a7a273dfe
Add HumanEval and GSM8K benchmarks to datasets ( #42648 )
...
Signed-off-by: southfreebird <yvorott@gmail.com >
Co-authored-by: Michael Goin <mgoin64@gmail.com >
2026-05-15 13:01:21 -07:00
Aaron Hao and GitHub
e0a45f1455
[Feat][RL] IPC weight sync optimizations: multigpu support and chunked packed tensors ( #37476 )
...
Signed-off-by: ahao-anyscale <ahao@anyscale.com >
Signed-off-by: hao-aaron <ahao@anyscale.com >
2026-05-15 22:53:06 +08:00
wang.yuqi and GitHub
75fd68c7a5
[Entrypoints] Split the pooling offline API into PoolingOfflineMixin. ( #42267 )
...
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
2026-05-15 08:05:57 +00:00
Cyrus Leung and GitHub
2676ab1e0b
[Deprecation] Remove old locations of get_tokenizer and resolve_hf_chat_template ( #35024 )
...
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk >
2026-05-15 00:13:32 -07:00
Louie Tsai and GitHub
e30f39c4f1
Update Intel Xeon model list and vLLM Benchmark Suite BKMs ( #42607 )
...
Signed-off-by: louie-tsai <louie.tsai@intel.com >
2026-05-15 05:14:03 +00:00
faa4b76afa
[Model] Support InternS2 Preview ( #42705 )
...
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn >
Co-authored-by: zxy <46674730+CUHKSZzxy@users.noreply.github.com >
2026-05-14 21:30:26 -07:00
aoshen02 and GitHub
8c79ad6580
Revert "[Core] Replace routing replay with device cache and async D2H pipeline" ( #39917 ) ( #42434 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
2026-05-13 23:49:01 -07:00
0d2732dd91
[MLA Attention Backend] Add TOKENSPEED_MLA backend for DSR1/Kimi K25 prefill + decode on Blackwell ( #41778 )
...
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
Signed-off-by: Roger Wang <hey@rogerw.io >
Co-authored-by: Roger Wang <hey@rogerw.io >
2026-05-13 23:48:02 -07:00
Michael Goin and GitHub
8efd508204
[Quantization] Rework quantization_config to use QuantKey and allow for activation override ( #41566 )
2026-05-13 16:58:32 -04:00
b3c69595a6
[MM][CG] Support ViT CG for Qwen2-VL ( #41736 )
...
Signed-off-by: John Calderon <jcalderon@nvidia.com >
Co-authored-by: Roger Wang <hey@rogerw.io >
2026-05-14 01:52:35 +08:00
CynicDora and GitHub
256dbcaabf
[Feature] Support custom callable proposer backend for speculative decoding ( #39487 )
...
Signed-off-by: 524031910363 <hyzhyzsh@sjtu.edu.cn >
Signed-off-by: CynicDora <hyzhyzsh@sjtu.edu.cn >
2026-05-13 16:53:01 +00:00
Shanshan Shen and GitHub
92def124bc
[MM][Perf][CG] Support ViT full CUDA graph for Qwen3.5 ( #42151 )
...
Signed-off-by: shen-shanshan <467638484@qq.com >
2026-05-13 16:00:32 +08:00
+4
ebeb09d822
[KV Transfer] Add MooncakeStoreConnector for KV cache offloading via Mooncake distributed store ( #40900 )
...
Signed-off-by: leichao.lc <leichao.lc@antgroup.com >
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: leichao.lc <leichao.lc@antgroup.com >
Co-authored-by: ivanium <yifanqiao@inferact.ai >
Co-authored-by: aoshen524 <aoshen@inferact.ai >
Co-authored-by: Dao007forever <daole@inferact.ai >
Co-authored-by: Teng Ma <sima.mt@alibaba-inc.com >
Co-authored-by: Pz1116 <zpbzpb123123@gmail.com >
Co-authored-by: foraxe <1055696449@qq.com >
Co-authored-by: Skywalker-EP <173423846@qq.com >
Co-authored-by: fems14 <1804143737@qq.com >
Co-authored-by: jianzs <zheng.shoujian@outlook.com >
Co-authored-by: baxingpiaochong <771405853@qq.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-05-12 16:09:10 -07:00
5a6a9fc6f6
[docs] Added one new contact to the Vulnerability Management team ( #42145 )
...
Signed-off-by: jperezde <jperezde@redhat.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-05-12 10:59:59 -04:00
bnellnm and GitHub
6427603ae8
[MoE Refactor] Move remaining experts classes to experts directory ( #42334 )
...
Signed-off-by: Bill Nell <bnell@redhat.com >
2026-05-12 09:19:46 -04:00
997132911e
[Doc] Fix typo in llm-d documentation link ( #42397 )
...
Signed-off-by: Florian Woerner <florian.woerner@onmyown.io >
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com >
2026-05-12 04:26:46 -07:00
28ee78af54
Implement custom dataset class for ASR benchmarking ( #41576 )
...
Signed-off-by: Yasmin Moslem <48152713+ymoslem@users.noreply.github.com >
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2026-05-12 12:17:58 +08:00
cf0d279142
[Docs] Add Apple Silicon documentation for vLLM-Metal GPU support ( #41987 )
...
Signed-off-by: alexagriffith <agriffith96@gmail.com >
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com >
2026-05-11 11:34:25 -07:00
5497ffbf7c
Add documentation about vLLM FIPS compliance ( #42190 )
...
Signed-off-by: Vinay Damodaran <vrdn@hey.com >
Signed-off-by: Vinay R Damodaran <vrdn@hey.com >
Co-authored-by: Russell Bryant <russell.bryant@gmail.com >
2026-05-11 18:17:02 +00:00
Nicolò Lucchesi and GitHub
770e9bd6b3
[Nixl][PD] Lease renewal TTL KV blocks on P ( #41383 )
...
Signed-off-by: NickLucche <nlucches@redhat.com >
2026-05-11 09:27:30 +00:00
Haoqing Wang and GitHub
5cba6839e6
Document MolmoWeb hf_overrides ( #42163 )
...
Signed-off-by: Haoqi Wang <78337154+hqhq1025@users.noreply.github.com >
2026-05-10 23:58:22 -07:00
wangxiyuan and GitHub
5536fc0c01
[Misc] Replace mamba_type string literals with MambaAttentionBackendEnum ( #41188 )
...
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com >
2026-05-11 03:59:36 +00:00
