dd0d74cd92
[Doc] Surface the --kv-cache-memory suggestion at INFO and document fast-startup knobs ( #47374 )
...
Signed-off-by: Nils Matteson <nils@thaw.sh >
Signed-off-by: Nils Matteson <nilsmatteson@icloud.com >
Co-authored-by: Nils Matteson <nils@thaw.sh >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-07-07 15:05:07 -07:00
Nick Cao and GitHub
47c40bfe8a
[Doc] Fix manylinux tag in installation guide ( #47913 )
...
Signed-off-by: Nick Cao <ncao@redhat.com >
2026-07-07 20:34:05 +00:00
93e2ab7111
Disable dynamic speculative decoding when DP is enabled ( #45963 )
...
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
2026-07-07 12:52:41 +00:00
Harry Mellor and GitHub
dd94484577
Bump Transformers version to 5.10.4 ( #41359 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-07-07 05:13:28 -07:00
cbb5f045be
[ROCm][CI] Refresh ROCm base images when docker rocm_base changes ( #46904 )
...
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com >
Signed-off-by: Codex <codex@example.invalid >
Signed-off-by: Micah Williamson <micah.williamson@amd.com >
Signed-off-by: Matthew Wong <Matthew.Wong2@amd.com >
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: Rohan138 <rohanpotdar138@gmail.com >
Co-authored-by: Codex <codex@example.invalid >
Co-authored-by: Micah Williamson <micah.williamson@amd.com >
Co-authored-by: Matthew Wong <Matthew.Wong2@amd.com >
2026-07-07 03:10:50 -07:00
Aritra Roy Gosthipaty and GitHub
1e823dc01d
[docs update] Update usage of hf cli for cache list and removal ( #47830 )
...
Signed-off-by: Aritra Roy Gosthipaty <aritra.born2fly@gmail.com >
2026-07-07 07:09:18 +00:00
700e882eab
Add TorchCodec as a video decoding backend ( #46609 )
...
Signed-off-by: Nicolas Hug <contact@nicolas-hug.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn >
2026-07-06 19:58:51 -07:00
86db6c3070
[Frontend] add per-request timing metrics field to response body of Chat/Completions APIs ( #46768 )
...
Signed-off-by: Nicholas Edelman <nedelman@nvidia.com >
Signed-off-by: Simon Mo <simon@inferact.ai >
Co-authored-by: Simon Mo <simon@inferact.ai >
Co-authored-by: GPT-5.5 <noreply@cursor.com >
2026-07-06 17:48:29 -07:00
5bce653e09
Make the Transformers modeling backend as fast as native vLLM ( #47187 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-07-06 16:59:14 +01:00
641cb59592
[Doc] Clarify fastokens availability ( #45813 )
...
Signed-off-by: LjjJzd <3542531707@qq.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-07-06 13:33:05 +00:00
Sungjae Lee and GitHub
3d7f357ebf
[Doc] docs: fix note formatting for pooling models ( #47701 )
...
Signed-off-by: Sungjae Lee <33976427+llsj14@users.noreply.github.com >
Signed-off-by: Sungjae Lee <sung-jae.lee@navercorp.com >
2026-07-06 09:01:10 +00:00
d9c1767cd4
[INC][ARK] Direct Register Custom Op for ARK ( #46361 )
...
Signed-off-by: Zhenzhong1 <zhenzhong.xu@intel.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-07-06 13:45:50 +08:00
Li, Jiang and GitHub
e9cc1fd093
[CI/Build][CPU] Remove global extra index ( #47687 )
...
Signed-off-by: jiang1.li <jiang1.li@intel.com >
2026-07-06 13:42:01 +08:00
95a248faed
[Attention Backend] HPC_ATTN backend support mtp and dynamic scheduled attention ( #47433 )
...
Signed-off-by: chengvjiang <chengvjiang@tencent.com >
Co-authored-by: chengvjiang <chengvjiang@tencent.com >
2026-07-05 18:18:25 -07:00
1a308c449c
[XPU] Add W8A8 FP8 linear kernel with multi-granularity quant support ( #43645 )
...
Signed-off-by: Chaojun,Zhang <chaojun.zhang@intel.com >
Signed-off-by: Chaojun Zhang <chaojun.zhang@intel.com >
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com >
2026-07-04 18:10:01 +08:00
07516fda67
[MRV2][SD] Make Dynamic SD comatible with Full Cuda Graphs ( #45953 )
...
Signed-off-by: Ekagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com >
Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
2026-07-03 23:58:27 -04:00
67ff0ae30f
Support nvfp4 kv with kv-cache-dtype-skip-layers sliding_window ( #42890 )
...
Signed-off-by: Shiyang Chen <shiychen@nvidia.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-07-04 02:29:13 +00:00
Kalyanam Dewri and GitHub
4875b4456b
[Doc] Fix VLM2Vec benchmark chat template path ( #47517 )
...
Signed-off-by: kalyanamdewri <kalyanampriyam@gmail.com >
2026-07-03 08:24:45 +00:00
d29125c085
Xqa decode kernels ( #43232 )
...
Signed-off-by: Dan Blanaru <48605845+DanBlanaru@users.noreply.github.com >
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com >
2026-07-02 12:32:05 -07:00
b0b8a286dd
[Model] Add LLaVA-OneVision-2 (LlavaOnevision2ForConditionalGeneration) ( #44785 )
...
Signed-off-by: chengzheng345 <209475443+chengzheng345@users.noreply.github.com >
Co-authored-by: chengzheng345 <209475443+chengzheng345@users.noreply.github.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-07-02 16:41:49 +08:00
3af8789559
[Feature] Universal speculative decoding for heterogeneous vocabularies (TLI) ( #38174 )
...
Signed-off-by: wan-danfeng <wandanfeng0802@gmail.com >
Signed-off-by: Wonderful <wandanfeng0802@gmail.com >
Co-authored-by: Wan_DF <wonderful199082@126.com >
Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com >
Co-authored-by: Benjamin Chislett <bchislett@nvidia.com >
2026-07-02 01:34:20 -07:00
xaguilar-amd and GitHub
09663abde0
[ROCm][MLA] Fuse MLA q/kv RMSNorm + FP8 per-token quant in the FP8 attention path ( #44977 )
...
Signed-off-by: Xavier Aguilar <xavier.aguilarfruto@amd.com >
Signed-off-by: Xavier Aguilar <Xavier.AguilarFruto@amd.com >
2026-07-02 13:00:41 +08:00
Harry Mellor and GitHub
a78c15616f
Migrate GPTBigCode and Starcoder2 to the Transformers modeling backend ( #30966 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-07-01 13:41:36 +00:00
5c4db60f01
docs(security): document gRPC interface as insecure for private use only ( #45903 )
...
Signed-off-by: jperezde <jperezde@redhat.com >
Signed-off-by: Russell Bryant <russell.bryant@gmail.com >
Co-authored-by: Russell Bryant <russell.bryant@gmail.com >
Co-authored-by: Russell Bryant <rbryant@redhat.com >
2026-07-01 12:39:57 +00:00
Harry Mellor and GitHub
a22e0dfc69
[Model] Remove AyaVision, MusicFlamingo ( #47263 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-07-01 10:39:33 +00:00
77a9c5ae28
Weight sync refactor + move sparse nccl engine ( #44353 )
...
Signed-off-by: hao-aaron <ahao@anyscale.com >
Signed-off-by: haoaaron <ahao@anyscale.com >
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com >
2026-07-01 01:25:19 -07:00
4470ae84de
Remove mantis ( #46806 )
...
Signed-off-by: Xianbao QIAN <xianbao.qian@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-07-01 07:13:58 +00:00
c5200d3565
[Attention][DSA] support dcp for FLASHINFER_MLA_SPARSE ( #46076 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Signed-off-by: Jingyi Yang <girasoleyang@gmail.com >
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Signed-off-by: GirasoleY <girasoleyang@gmail.com >
Co-authored-by: Jingyi Yang <girasoleyang@gmail.com >
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
2026-07-01 00:32:20 -04:00
Harry Mellor and GitHub
5dc36a4fa5
[Model] Remove Tarsier, Tarsier2 ( #47143 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-06-30 13:20:33 +00:00
8cf7c4d8ad
[Attention Backend] add HPC-Ops Attention backend ( #46020 )
...
Signed-off-by: chengvjiang <chengvjiang@tencent.com >
Co-authored-by: chengvjiang <chengvjiang@tencent.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-06-30 18:17:43 +08:00
Juan Pérez de Algaba and GitHub
364ee36af1
fix(security): prevent image decompression bomb OOM denial of service ( #47010 )
...
Signed-off-by: jperezde <jperezde@redhat.com >
2026-06-30 09:39:22 +00:00
bec232a914
Secondary tier implementation for PD disaggregation ( #42285 )
...
Signed-off-by: Liran Schour <lirans@il.ibm.com >
Signed-off-by: liranschour <liranschour@users.noreply.github.com >
Co-authored-by: Or Ozeri <or@ozery.com >
Co-authored-by: Or Ozeri <oro@il.ibm.com >
2026-06-30 07:51:44 +03:00
b5c9e1ac33
[LoRA] Add language-backbone LoRA support for MiniCPM-V 4.6 ( #46740 )
...
Signed-off-by: linitra24 <Joy25810@foxmail.com >
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com >
2026-06-30 04:19:31 +00:00
Woosuk Kwon and GitHub
a309d4fe60
Support DCP with FlashInfer MLA ( #43729 )
...
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
2026-06-29 13:24:29 -07:00
wang.yuqi and GitHub
3483240b7e
[Frontend] Consolidate scale out entrypoints ( #44512 )
...
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
2026-06-29 03:18:53 -07:00
Harry Mellor and GitHub
f6bb8682ee
Fix docs on main ( #47009 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-06-29 15:50:57 +08:00
4559c43a95
[MM][CG] Gemma3 Encoder CUDA Graph ( #43591 )
...
Signed-off-by: JisoLya <523420504@qq.com >
Signed-off-by: Soyaazz <523420504@qq.com >
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn >
2026-06-29 04:52:00 +00:00
4dfbf1503b
[Model] Add support for openai/privacy-filter ( #41026 )
...
Signed-off-by: Fabian Joswig <fjosw@users.noreply.github.com >
Co-authored-by: wang.yuqi <yuqi.wang@daocloud.io >
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com >
2026-06-28 16:18:22 -07:00
35e6c86caa
[Bugfix][MM][CG] Enable dual-path ViT CUDA graph for Step3-VL ( #46034 )
...
Signed-off-by: shen-shanshan <467638484@qq.com >
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn >
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn >
2026-06-28 00:06:43 -07:00
c6741b2ad4
[Model] Support Unlimited OCR ( #46564 )
...
Signed-off-by: Tianyu Guo <guoty9@mail2.sysu.edu.cn >
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn >
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn >
Co-authored-by: Roger Wang <hey@rogerw.io >
2026-06-27 23:09:18 -07:00
Benjamin Chislett and GitHub
455f25aa13
[CLI] Add flag to print TTFT and TPS in vllm chat ( #46775 )
...
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com >
2026-06-26 22:15:10 -07:00
ddd3855a28
[MoE Backend] add HPC-Ops MoE backend ( #45924 )
...
Signed-off-by: chengvjiang <chengvjiang@tencent.com >
Co-authored-by: chengvjiang <chengvjiang@tencent.com >
Co-authored-by: youkaichao <youkaichao@gmail.com >
2026-06-27 11:18:07 +08:00
c6554f321c
[CPU] Fix macOS/Apple Silicon hang by enabling OpenMP in the build ( #46769 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com >
2026-06-26 14:32:21 -04:00
d980a3cc6e
[ROCm] Fix AITER_UNIFIED_ATTN Dispatching After AITER Bump ( #46780 )
...
Signed-off-by: Micah Williamson <micah.williamson@amd.com >
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com >
Co-authored-by: Rohan138 <rohanpotdar138@gmail.com >
2026-06-26 02:09:56 -07:00
bf292b5f6b
[Docs] Remove BambaForCausalLM from supported hybrid models list ( #46071 )
...
Signed-off-by: liejiang <jianglie2023@gmail.com >
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com >
2026-06-26 08:02:50 +00:00
wang.yuqi and GitHub
5e3dad04b1
[Misc] Move the legacy api_server.py to the examples directory. ( #46783 )
...
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
2026-06-26 07:43:29 +00:00
Tiezhen WANG and GitHub
c7645bce04
Remove grok model arch from vllm ( #46706 )
...
Signed-off-by: Xianbao QIAN <xianbao.qian@gmail.com >
2026-06-25 23:02:10 -07:00
72adb20a6a
[Model] Remove AquilaForCausalLM, AquilaModel ( #46605 )
...
Signed-off-by: Xianbao QIAN <xianbao.qian@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-06-25 08:08:26 +00:00
2396d91e93
[CPU][Spec Decode] Enable DFlash SD for CPU ( #44029 )
...
Signed-off-by: guybd <guy.boudoukh@intel.com >
Signed-off-by: Guy Boudoukh <guy.boudoukh@intel.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-06-25 15:32:48 +08:00
d511b5bae9
Chore: Fix minor doc sentence, grammar, quote errors ( #40469 )
...
Signed-off-by: Ashwin Phadke <23502062+ashwin-phadke@users.noreply.github.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-06-24 18:58:24 +00:00
