d6aeaddf4a
[bugfix] fix type[AttentionBackend] bug in kv_connector_base_v1 ( #30051 )
...
Signed-off-by: 01267596 <xiongkai123@cmbchina.com >
Co-authored-by: 01267596 <xiongkai123@cmbchina.com >
2025-12-06 07:11:31 +00:00
Woosuk Kwon and GitHub
a238cbd89d
[Model Runner V2] Support min-p sampling ( #30171 )
...
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu >
2025-12-05 21:42:47 -08:00
Nick Hill and GitHub
4026ae31e9
[Misc] Move disable_nccl_for_dp_synchronization init logic into VllmConfig ( #30161 )
...
Signed-off-by: Nick Hill <nhill@redhat.com >
2025-12-05 20:59:04 -08:00
b12f4a9830
[CI/Build][AMD] Use ROCM_ATTN instead of FLASH_ATTN test for test_register_kv_caches for ROCm and update test for TRITON_ATTN ( #29985 )
...
Signed-off-by: Randall Smith <ransmith@amd.com >
Co-authored-by: Randall Smith <ransmith@amd.com >
Co-authored-by: TJian <tunjian.tan@embeddedllm.com >
2025-12-05 20:57:38 -08:00
Rohan Potdar and GitHub
40a046cd82
[Bugfix]: Fix TokenizerLike interface ( #30009 )
...
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com >
2025-12-05 20:56:40 -08:00
Peter Salas and GitHub
e858bc4d14
[Model] Add support for transformer-based Ultravox v0.7 projector ( #30089 )
...
Signed-off-by: Peter Salas <peter@fixie.ai >
2025-12-05 20:55:43 -08:00
Dongjie Zou and GitHub
e3fbb6f152
fix#30092 Kimi-Linear model loading failure with missing indexer_rotary_emb ( #30093 )
...
Signed-off-by: baonudesifeizhai <baonudesifeizhai@gmail.com >
2025-12-05 20:55:09 -08:00
yuttian1 and GitHub
c4d62618ca
Fix AWQ MoE marlin check issue in marlin_utils.py for AMD backend ( #30102 )
...
Signed-off-by: yuttian1 <yuttian@amd.com >
2025-12-05 20:54:38 -08:00
62079d8600
[CI/Build][AMD] Skip marlin, machete, and hadacore tests since these require _C functions not defined for ROCm ( #30109 )
...
Signed-off-by: Randall Smith <ransmith@amd.com >
Co-authored-by: Randall Smith <ransmith@amd.com >
2025-12-06 12:54:17 +08:00
Harry Mellor and GitHub
bf4a901af9
Better error when world size is larger than node and distributed_executor_backend is not set ( #30140 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2025-12-05 20:53:52 -08:00
7e31c3a3f6
[CI]: Remove unnecessary imports from test_lmache_integration ( #30157 )
...
Signed-off-by: Samuel Shen <slshen@uchicago.edu >
Co-authored-by: Samuel Shen <slshen@uchicago.edu >
2025-12-06 12:53:34 +08:00
dc839ad03d
[CI/Build][AMD][Quantization] Fix test_int8_kernel.py by updating int8_utils to use hip.libdevice.round ( #30151 )
...
Signed-off-by: Randall Smith <ransmith@amd.com >
Co-authored-by: Randall Smith <ransmith@amd.com >
2025-12-05 20:52:11 -08:00
Deboleina and GitHub
02a4169193
[Tests] Tool call tests for openai/gpt-oss-20b ( #26237 )
...
Signed-off-by: Debolina Roy <debroy@redhat.com >
2025-12-05 19:03:29 -08:00
Wentao Ye and GitHub
7b5575fa7d
[Bug] Fix vLLM config is not set error ( #29999 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2025-12-05 16:42:12 -05:00
Bangsheng Tang and GitHub
77e4472809
let draft model follow target model's config_format ( #30152 )
2025-12-05 13:33:42 -08:00
Divakar Verma and GitHub
962d703818
[Bugfix][llama4_eagle] Fix missing 'lm_head' attribute ( #29926 )
...
Signed-off-by: Divakar Verma <divakar.verma@amd.com >
2025-12-05 19:57:26 +00:00
Nicolò Lucchesi and GitHub
e23ca3a0e8
[CI] Re-use whisper_client for all tests ( #30148 )
...
Signed-off-by: NickLucche <nlucches@redhat.com >
2025-12-05 19:47:37 +00:00
Russell Bryant and GitHub
3633035a3f
[Misc] Rename CohereForAI references to CohereLabs ( #30147 )
...
Signed-off-by: Russell Bryant <rbryant@redhat.com >
2025-12-05 19:41:40 +00:00
Nicolò Lucchesi and GitHub
bff78310d9
[Enc-Dec] Fix OOT tokenizer issue ( #30144 )
...
Signed-off-by: NickLucche <nlucches@redhat.com >
2025-12-05 19:23:33 +00:00
adb315060c
[KVConnector][Feature] Support KV connector cache reset via /reset_prefix_cache ( #27170 )
...
Signed-off-by: tovam <tovam@pliops.com >
Signed-off-by: Tova Movshovitz <tovam@pliops.com >
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2025-12-05 18:33:26 +00:00
4e26d3b09e
[Compile] Conditional compilation. Introduce compile_ranges ( #24252 )
...
Signed-off-by: Luka Govedič <lgovedic@redhat.com >
Signed-off-by: Luka Govedič <ProExpertProg@users.noreply.github.com >
Signed-off-by: ilmarkov <markovilya197@gmail.com >
Signed-off-by: Luka Govedič <luka.govedic@gmail.com >
Signed-off-by: ProExpertProg <lgovedic@redhat.com >
Co-authored-by: Luka Govedič <lgovedic@redhat.com >
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com >
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
Co-authored-by: Luka Govedič <luka.govedic@gmail.com >
2025-12-05 18:17:32 +00:00
66e674cdd5
[Attention][UX][1/N] Add AttentionConfig and change attention env vars to CLI arguments ( #26315 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Signed-off-by: Matthew Bonanni <mbonanni001@gmail.com >
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
2025-12-05 09:48:43 -08:00
Mark McLoughlin and GitHub
dff0a2b394
[NIXL] Add remote_request_id to kv_transfer_params ( #29665 )
...
Signed-off-by: Mark McLoughlin <markmc@redhat.com >
2025-12-05 09:43:48 -08:00
Nick Hill and GitHub
dc264bcea1
[BugFix] Eagerly abort cancelled final-step requests ( #29987 )
...
Currently, when requests are cancelled while executing their final
step, "completion" is handled based on normal stop processing (e.g.
length or stop token), so the abort has no effect. This is typically
not a problem, but when a kv connector is involved it thinks the
request completed successfully rather than being aborted.
This is problematic for disaggregated prefill which will free kv
cache blocks if the request was aborted but not if it completed
successfully—since the cancelled request will never be sent to
the decode side, kv cache blocks remain pinned until the fall-back
timeout expires. The problem is exacerbated when many requests
are cancelled and/or there are large prefills whose forward pass
takes a long time (since the window is bigger).
This PR fixes the problem by processing pending aborts
immediately prior to processing model output each step; we process
only aborts, not new requests, since it's preferable for latency to
process model outputs before new incoming requests.
Fixes #26400 .
Signed-off-by: Nick Hill <nhill@redhat.com >
2025-12-05 17:28:32 +00:00
78c44fd722
[NIXL] Small cleanup of unused variables ( #29618 )
...
Signed-off-by: NickLucche <nlucches@redhat.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2025-12-05 18:17:36 +01:00
Angela Yi and GitHub
e7296b08da
[bugfix] Pass globals to aot_compiled function ( #29428 )
...
Signed-off-by: angelayi <yiangela7@gmail.com >
2025-12-05 16:54:26 +00:00
da7bc54ea8
[responsesAPI][5] ResponsesParser with tools for full MCP python loop ( #29798 )
...
Signed-off-by: Andrew Xia <axia@fb.com >
Signed-off-by: Andrew Xia <axia@meta.com >
Co-authored-by: Andrew Xia <axia@fb.com >
2025-12-05 11:11:50 -05:00
Mark McLoughlin and GitHub
949a6a19d2
[NIXL] Add compatibility checking to NIXL KV connector handshake ( #29503 )
...
Signed-off-by: Mark McLoughlin <markmc@redhat.com >
2025-12-05 15:52:45 +01:00
2c174420f5
Reduce validation to a warning ( #28749 )
...
Signed-off-by: Alec Solder <alecs@fb.com >
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Alec Solder <alecs@fb.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2025-12-05 14:02:49 +00:00
Yi Liu and GitHub
0d8a7d8a26
[Compressed Tensors] Add XPU wNa16 support ( #29484 )
...
Signed-off-by: yiliu30 <yi4.liu@intel.com >
2025-12-05 22:02:09 +08:00
9843e332da
[CPU][Perf] Add fast vectorized exp impl from Arm Optimized Routines ( #30068 )
...
Signed-off-by: Ubuntu <ubuntu@ip-10-252-30-150.eu-west-1.compute.internal >
Signed-off-by: Elham Harirpoush <elham.harirpoush@arm.com >
Co-authored-by: Ubuntu <ubuntu@ip-10-252-30-150.eu-west-1.compute.internal >
2025-12-05 13:09:20 +00:00
Harry Mellor and GitHub
b7d85cf25c
[CI] Have pre-commit comment on a PR if pre-commit was not used ( #30077 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2025-12-05 13:03:45 +00:00
c2894d3883
[Feature] Add Layer-wise NVTX Support ( #29990 )
...
Signed-off-by: Max Hu <hyoung2991@gmail.com >
Signed-off-by: Max Hu <maxhu@nvidia.com >
Co-authored-by: Max Hu <maxhu@nvidia.com >
2025-12-05 11:20:07 +00:00
Zhiwei and GitHub
3628bcaaf2
[ROCm][MXFP4] Infer w4a4 quant method in rocm aiter fused moe ( #29775 )
...
Signed-off-by: ZhiweiYan-96 <zhiwei.yan@amd.com >
2025-12-05 11:01:16 +00:00
strinczer and GitHub
b73b158ab0
[Bugfix] Fix parse_output_message crash on commentary with no recipient ( #29972 )
...
Signed-off-by: Shai Trinczer <strinczer@icloud.com >
Signed-off-by: strinczer <strinczer@icloud.com >
2025-12-05 10:51:12 +00:00
Ning Xie and GitHub
7ae13c66ba
[typing] fix type ( #29964 )
...
Signed-off-by: Andy Xie <andy.xning@gmail.com >
2025-12-05 10:46:08 +00:00
Ming Yang and GitHub
f16356fe36
[bench] Support common prefix len config (for decode-only bench) ( #29934 )
...
Signed-off-by: Ming Yang <minos.future@gmail.com >
2025-12-05 10:26:52 +00:00
65ee97288a
[BugFix] Adding env variable to disable async grammar compilation ( #29996 )
...
Signed-off-by: Alec Solder <alecs@fb.com >
Signed-off-by: Alec S <10566873+alecsolder@users.noreply.github.com >
Co-authored-by: Alec Solder <alecs@fb.com >
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: 22quinn <33176974+22quinn@users.noreply.github.com >
2025-12-05 00:49:37 -08:00
Yanan Cao and GitHub
62b3333448
[Frontend] Remove deprecated -O.xx flag ( #29991 )
...
Signed-off-by: Yanan Cao <gmagogsfm@gmail.com >
2025-12-05 00:47:22 -08:00
feecba09af
[CI/Build][AMD] Use float16 in test_reset_prefix_cache_e2e to avoid accuracy issues ( #29997 )
...
Signed-off-by: Randall Smith <ransmith@amd.com >
Co-authored-by: Randall Smith <ransmith@amd.com >
2025-12-05 08:42:25 +00:00
amitz-nv and GitHub
6038b1b04b
[Frontend][Model] Add 'float16' to possible mamba cache dtype values, override mamba SSM cache dtype value for NemotronH ( #29978 )
...
Signed-off-by: amitz-nv <203509407+amitz-nv@users.noreply.github.com >
2025-12-05 00:34:33 -08:00
Tiger Xu / Zhonghu Xu and GitHub
60a66ea2dc
[DOC]: Add kthena to integrations ( #29931 )
...
Signed-off-by: Zhonghu Xu <xuzhonghu@huawei.com >
2025-12-05 08:11:03 +00:00
Micah Williamson and GitHub
06579f9a82
[AMD][CI] Add ray[default] Dependency On ROCm To Pass v1/metrics/test_engine_logger_apis.py ( #30110 )
...
Signed-off-by: Micah Williamson <micah.williamson@amd.com >
2025-12-05 06:48:23 +00:00
Chukwuma Nwaugha and GitHub
6e865b6a83
Refactor example prompts fixture ( #29854 )
...
Signed-off-by: nwaughac@gmail.com
2025-12-05 06:44:32 +00:00
d698bb382d
[Bugfix] Correct num_q_heads on DCP for Flashinfer backends ( #29487 )
...
Signed-off-by: Jingchun Gao <gaojingchun1@huawei.com >
Signed-off-by: Jingchun Gao <63247409+gjc0824@users.noreply.github.com >
Co-authored-by: Jingchun Gao <gaojingchun1@huawei.com >
2025-12-05 05:54:31 +00:00
Charlie Fu and GitHub
2c22c4ca2d
[ROCm][CI] Increase the memory threshold for test_deep_sleep_fp8_kvcache ( #30104 )
...
Signed-off-by: charlifu <charlifu@amd.com >
2025-12-05 04:51:44 +00:00
Laith Sakka and GitHub
5867819eaf
Do not guard during noop elimination pass ( #30095 )
...
Signed-off-by: Laith Sakka <lsakka@meta.com >
2025-12-05 04:10:12 +00:00
Charlie Fu and GitHub
7c9b2c8f81
[ROCm][CI] Add jiwer dependency for testing ( #30081 )
...
Signed-off-by: charlifu <charlifu@amd.com >
2025-12-05 03:34:51 +00:00
0098a6e3da
[PCP&DCP] move CUDAGraph check for PCP&DCP to the check func of platforms ( #29952 )
...
Signed-off-by: QiuChunshuo <qiuchunshuo@huawei.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2025-12-04 21:40:51 -05:00
Hubert de La Jonquiere and GitHub
befb59e5b1
[Model] Add Holo2 reasoning parser ( #30048 )
...
Signed-off-by: hdlj-h <hubert@hcompany.ai >
2025-12-05 10:38:45 +08:00
