|
a9a8a32dcd
|
Register parsed config classes before tokenizer init (#40299)
Signed-off-by: Bortlesboat <bortstheboat@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-06-16 05:33:08 +00:00 |
|
|
|
9d808e2309
|
[Core] Use fastsafetensors ParallelLoader for weight loading (#40183)
Signed-off-by: Git Bisector <gitbisector@gmail.com>
Signed-off-by: gitbisector <gitbisector@gmail.com>
Signed-off-by: git bisector <gitbisector@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
|
2026-06-15 22:32:05 -07:00 |
|
|
Ben BrowningandGitHub
|
f3858d5422
|
[Frontend] [Parser] Migrate Nemotron V3 to streaming parser engine (#45755)
Signed-off-by: Ben Browning <bbrownin@redhat.com>
|
2026-06-16 05:31:21 +00:00 |
|
|
Bugen ZhaoandGitHub
|
259ff891be
|
[Rust Frontend] Require ModelConfig.vocab_size to be present (#45696)
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-06-16 05:30:25 +00:00 |
|
|
|
6607a80dab
|
[Bugfix][Gemma4] Fix offline parser truncation, adjust_request token leak, and chat template sync (#45553)
Signed-off-by: Luciano Martins <lucianommartins@users.noreply.github.com>
Co-authored-by: Luciano Martins <lucianommartins@users.noreply.github.com>
|
2026-06-16 04:31:53 +00:00 |
|
|
liuzhenweiandGitHub
|
b8bd773fe4
|
[XPU] Fix Triton attn fp8/bf16 check failing (#45758)
Signed-off-by: zhenwei-intel <zhenwei.liu@intel.com>
|
2026-06-16 12:31:20 +08:00 |
|
|
Ruinan MaandGitHub
|
2addbb9cc9
|
[BugFix] Support async scheduling with prompt embeds for multimodal models (#45673)
Signed-off-by: Ruinan Ma <r7ma3088@gmail.com>
|
2026-06-16 04:12:54 +00:00 |
|
|
Isotr0pyandGitHub
|
e3cfea2e1b
|
[Multimodal] Add Qwen3-VL video loader (#44412)
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
|
2026-06-16 03:45:34 +00:00 |
|
|
Bugen ZhaoandGitHub
|
f99260d2aa
|
[Rust Frontend] Lower out-of-vocab validation to text layer (#45685)
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-06-16 03:37:58 +00:00 |
|
|
Bugen ZhaoandGitHub
|
3f65e21e32
|
[Rust Frontend] Support max_logprobs validation (#45674)
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-06-16 10:57:56 +08:00 |
|
|
xx-thomasandGitHub
|
b00e76ff72
|
[Misc][Model] add io processor for query/document embeddings from ColBERT (jinaai/jina-colbert-v2) (#45210)
Signed-off-by: thomas <thomas.varghese@columbia.edu>
|
2026-06-16 01:32:32 +00:00 |
|
|
Woosuk KwonandGitHub
|
f4359a70f9
|
[DSV4][Minor] Fix supported KV cache dtypes (#44892)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
|
2026-06-16 00:14:51 +00:00 |
|
|
Itay AlroyandGitHub
|
3afe659b6b
|
[EP] Enable DBO with NIXL EP (#45275)
Signed-off-by: Itay Alroy <ialroy@nvidia.com>
|
2026-06-15 23:37:22 +00:00 |
|
|
Itay AlroyandGitHub
|
16e91176cf
|
[EP] Query NIXL EP top-k index dtype (#45298)
Signed-off-by: Itay Alroy <ialroy@nvidia.com>
|
2026-06-15 22:50:18 +00:00 |
|
|
Itay AlroyandGitHub
|
ab8b0fe338
|
nixl_ep: Skip post-receive quantization for NVFP4 (#45606)
Signed-off-by: Itay Alroy <ialroy@nvidia.com>
|
2026-06-15 22:42:05 +00:00 |
|
|
|
d467a2a7f2
|
[Bugfix] Defer block freeing until in-flight steps finish under async scheduling + PD KV consumer (#45357)
Signed-off-by: llx-08 <2596671364@qq.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com>
|
2026-06-15 21:36:09 +00:00 |
|
|
|
76a373eff4
|
[Frontend] Replace legacy Gemma4 parsers with engine-based implementation (#45588)
Signed-off-by: Ben Browning <bbrownin@redhat.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
|
2026-06-15 21:34:07 +00:00 |
|
|
Zang PeiyuandGitHub
|
25ee659db0
|
Fix parallel_tool_calls: null treated as false instead of default true (#44955)
Signed-off-by: factnn <166481866+factnn@users.noreply.github.com>
|
2026-06-15 21:14:10 +00:00 |
|
|
|
eacff17c8d
|
[Model Runner V2][Bugfix] Fix MRV2 LoRA warmup (#35536)
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>
|
2026-06-15 13:17:23 -07:00 |
|
|
Flora FengandGitHub
|
cd9078fe59
|
[Frontend] Skip structural tags for auto tool_choice without strict mode (#45600)
Signed-off-by: sfeng33 <4florafeng@gmail.com>
|
2026-06-15 19:55:31 +00:00 |
|
|
Wentao YeandGitHub
|
e18fe932ca
|
[Perf] Optimize DSv4 prefill chunk planning, 4.0% E2E Throughput Improvement (#45061)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
|
2026-06-15 19:50:21 +00:00 |
|
|
|
51ec5cf08f
|
[Bugfix] Chat Completions Harmony Refactor Clean up (#45464)
Signed-off-by: Yifan Zong <yzong@redhat.com>
Co-authored-by: Ben Browning <bbrownin@redhat.com>
|
2026-06-15 14:45:19 -04:00 |
|
|
|
7e612a0f06
|
[KV Offloading] Implement reset_cache for TieringOffloadingManager (#44541)
Signed-off-by: Ronen Schaffer <ronen.schaffer@ibm.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-06-15 18:42:53 +00:00 |
|
|
+1
|
0a1c5034f5
|
[Model] Add MiniMax M3 support (#45381)
Signed-off-by: youkaichao <youkaichao@gmail.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Signed-off-by: functionstackx <47992694+functionstackx@users.noreply.github.com>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: functionstackx <47992694+functionstackx@users.noreply.github.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai>
|
2026-06-16 01:01:25 +08:00 |
|
|
RoyWangandGitHub
|
a3195fab7b
|
[AMD][Bugfix][Quantization] Honor fused-name match in is_layer_skipped (#43981)
|
2026-06-15 09:37:52 -07:00 |
|
|
Flora FengandGitHub
|
0d80979644
|
[Chore] Consolidate reasoning/tool parser attributes into unified Parser in chat serving (#45548)
Signed-off-by: sfeng33 <4florafeng@gmail.com>
|
2026-06-15 11:16:45 -04:00 |
|
|
SaddssandGitHub
|
588db18362
|
[Bugfix] Two-phase KV allocation for cross-group prefix cache hits (supersedes #33775) (#44409)
Signed-off-by: Saddss <2872669061@qq.com>
|
2026-06-15 22:39:59 +08:00 |
|
|
|
fa63bb9db6
|
Remove redundant Triton KV cache dtype asserts and enforce architectural support (fp8 >= sm89) (#43914)
Signed-off-by: Mike G <180722391+mikekg@users.noreply.github.com>
Co-authored-by: Michael Gschwind <mgschwind@nvidia.com>
|
2026-06-15 06:49:57 -07:00 |
|
|
|
5ed15f42b9
|
Fix the E8M0 scale computation in the MXFP4 (W4A4) MOE CUTLASS kernel (#43557)
Signed-off-by: Xin He <xin3.he@intel.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
|
2026-06-15 06:04:54 -07:00 |
|
|
Juan Pérez de AlgabaandGitHub
|
b997071ec4
|
(security) Enforce audio upload size limit before full file materialization (#45510)
Signed-off-by: jperezde <jperezde@redhat.com>
|
2026-06-15 10:25:24 +00:00 |
|
|
Martin KuklaandGitHub
|
6c5872efc5
|
[Bugfix] Unset HF's default max_new_tokens for DiffusionGemma (#45417)
Signed-off-by: Martin Kukla <martin.kukla@cantab.net>
|
2026-06-15 17:31:57 +08:00 |
|
|
wang.yuqiandGitHub
|
1d88c4dadd
|
[Docs] Update the online serving docs. (#45676)
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
|
2026-06-15 17:23:36 +08:00 |
|
|
vllmellmandGitHub
|
25c53d1293
|
[ROCm][Doc] Add installation notes about python version requirement (#45671)
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
|
2026-06-15 17:22:55 +08:00 |
|
|
Yejing LaiandGitHub
|
9872921c5f
|
[XPU] skip UT test_with_ngram_gpu_spec_decoding (#44423)
Signed-off-by: Lai, Yejing <yejing.lai@intel.com>
|
2026-06-15 08:46:30 +00:00 |
|
|
ReidandGitHub
|
c17e2f7c84
|
[Bugfix][Rust Frontend] Make metrics respect --served-model-name (#45465)
Signed-off-by: reidliu41 <reid201711@gmail.com>
|
2026-06-15 08:05:10 +00:00 |
|
|
FAUSTandGitHub
|
40eac9a9d9
|
[Rust Frontend] Support parallel_tool_calls = false (#44760)
Signed-off-by: zhoujinyu <2319109590@qq.com>
|
2026-06-15 07:50:48 +00:00 |
|
|
|
b5adb027ad
|
[Models] Fix MiMo v2.x QKV TP sharding + FP4 support (#45200)
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-06-15 15:13:34 +08:00 |
|
|
Sahil SinghandGitHub
|
64833f8158
|
[Rust Frontend] Add external→internal request-id map for abort() (#45137)
Signed-off-by: Sahil Singh <sahiilsiingh37@gmail.com>
|
2026-06-15 06:51:24 +00:00 |
|
|
|
ddad5dbda2
|
[Bugfix][Rust] Sync EngineCoreReadyResponse with the Python dataclass (#45557)
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Will Eaton <weaton@redhat.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-06-15 06:49:42 +00:00 |
|
|
Peter PanandGitHub
|
ebb0a71ad0
|
[Bugfix] Reject out-of-range temperature values in SamplingParams (#44965)
Signed-off-by: Peter Pan <Peter.Pan@daocloud.io>
|
2026-06-14 23:12:44 -07:00 |
|
|
Ting SUNandGitHub
|
48df95c43e
|
[Feature][Frontend] Report multimodal token counts in usage.prompt_tokens_details (#45458)
Signed-off-by: Ting Sun <suntcrick@gmail.com>
|
2026-06-15 05:20:58 +00:00 |
|
|
|
7df4fe1bd7
|
[Model] Remove XverseForCausalLM (#45638)
Signed-off-by: Xianbao QIAN <xianbao.qian@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-06-14 22:09:00 -07:00 |
|
|
|
b8336c3c7c
|
[Bugfix][V1] Split V2 model-runner attention groups on num_heads_q (#45564)
Signed-off-by: Roger Wang <hey@rogerw.io>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-06-14 21:49:46 -07:00 |
|
|
|
e8d3e22c88
|
Fix included router missing path for FastAPI >=0.137 (#45629)
Signed-off-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
|
2026-06-15 04:28:52 +00:00 |
|
|
|
c4a3f9d137
|
[Frontend] Add Streaming Parser Engine and new Qwen3 Parser (#45413)
Signed-off-by: Ben Browning <bbrownin@redhat.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
|
2026-06-15 11:59:05 +08:00 |
|
|
Flora FengandGitHub
|
e3e3cd5458
|
[Bugfix][CI] Update Dockerfile dependency graph PNG (#45602)
Signed-off-by: sfeng33 <4florafeng@gmail.com>
v0.23.1rc0
|
2026-06-15 10:35:24 +08:00 |
|
|
Li, JiangandGitHub
|
8760f972ca
|
[CPU] Refine CPU attention frontend (#45391)
Signed-off-by: jiang1.li <jiang1.li@intel.com>
|
2026-06-14 19:26:54 -07:00 |
|
|
|
b675cb7d0f
|
[Bugfix][CPU] Honor cgroup memory limit when computing KV cache size (#45086)
Signed-off-by: baoloongmao <baoloongmao@tencent.com>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>
|
2026-06-14 19:26:50 -07:00 |
|
|
Chaojun ZhangandGitHub
|
2725c84aae
|
[XPU] Enable sequence parallel support for XPU (#38608)
Signed-off-by: chaojun-zhang <chaojun.zhang@intel.com>
Signed-off-by: Chaojun Zhang <chaojun.zhang@intel.com>
Signed-off-by: Chaojun,Zhang <chaojun.zhang@intel.com>
|
2026-06-14 19:26:46 -07:00 |
|
|
Noa NeriaandGitHub
|
1801fad0ba
|
[Bugfix] Stream Llama4 weight loading to avoid host-OOM with copy-returning loaders (#44645)
Signed-off-by: Noa Neria <nneria@nvidia.com>
|
2026-06-14 19:23:44 -07:00 |
|