e97c3cb303
[Core] Persist and reuse the memory-profiling result across boots (opt-in) ( #47388 )
...
Signed-off-by: Nils Matteson <nils@thaw.sh >
Signed-off-by: Nils Matteson <nilsmatteson@icloud.com >
Co-authored-by: Nils Matteson <nils@thaw.sh >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-07-08 00:53:02 +00:00
Juan Pérez de Algaba and GitHub
675f4295cd
fix(security): bound completion prompt list to prevent unbounded engine fan-out ( #47845 )
...
Signed-off-by: jperezde <jperezde@redhat.com >
2026-07-07 22:48:20 +00:00
Jason Li and GitHub
d99adcebdc
[BugFix] Fix ModelOpt quantization inference for fused siblings ( #47445 )
...
Signed-off-by: jasonlizhengjian <jasonlizhengjian@gmail.com >
2026-07-08 03:19:42 +05:00
55da232db6
[Bugfix] Pad Mamba page size instead of scaling block_size in unify_kv_cache_spec_page_size ( #45207 )
...
Signed-off-by: Sahil170595 <147995121+Sahil170595@users.noreply.github.com >
Co-authored-by: Claude Fable 5 <noreply@anthropic.com >
2026-07-07 22:01:34 +00:00
Rishabh Saini and GitHub
2f3f441f84
fix: include topic frame in KV events replay response ( #45177 )
...
Signed-off-by: RishabhSaini <rishabhsaini01@gmail.com >
2026-07-07 14:48:23 -04:00
d6875196ad
[Bugfix] Exclude kv_cache_memory_bytes from CacheConfig.compute_hash ( #47356 )
...
Signed-off-by: Nils Matteson <nils@thaw.sh >
Signed-off-by: Nils Matteson <nilsmatteson@icloud.com >
Co-authored-by: Nils Matteson <nils@thaw.sh >
2026-07-07 10:46:51 -07:00
bdc6f3bfa1
[Bug] Fix tmp directory for lm_eval ( #47755 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-07-07 16:40:12 +00:00
392d1b4d2e
[BugFix][LoRA] Refresh punica metadata when LoRA slots are reassigned under an unchanged mapping ( #47725 )
...
Signed-off-by: AmeenP <ameenp360@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-07-07 08:53:55 -07:00
c46ced1ee3
[kv_offload] Establish tier-owned KV event handling ( #46544 )
...
Signed-off-by: Change72 <changg@nvidia.com >
Signed-off-by: Chang Guo <changg@nvidia.com >
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
Co-authored-by: Or Ozeri <oro@il.ibm.com >
2026-07-07 16:55:20 +03:00
65dcde1695
[Bugfix] Fix PD disagg + MTP correctness for Qwen3.5(GDN) ( #47466 )
...
Signed-off-by: Dakai An <dakaian108@gmail.com >
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
2026-07-07 13:51:22 +00:00
65a7b46284
[KV-Offloading] Support workload identity for objectstore secondary tier ( #47063 )
...
Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com >
Co-authored-by: Or Ozeri <oro@il.ibm.com >
2026-07-07 16:30:16 +03:00
93e2ab7111
Disable dynamic speculative decoding when DP is enabled ( #45963 )
...
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
2026-07-07 12:52:41 +00:00
8b91cd5b20
[Bugfix][Core] Close underlying iterator in merge_async_iterators single-iterator fast path ( #44726 )
...
Signed-off-by: Ting Sun <suntcrick@gmail.com >
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
2026-07-07 05:13:41 -07:00
Harry Mellor and GitHub
dd94484577
Bump Transformers version to 5.10.4 ( #41359 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-07-07 05:13:28 -07:00
danielafrimi and GitHub
0a2965b1b3
[BugFix] Fix ModelOpt mixed-precision quantization for sparse quantized_layers configs. ( #47318 )
...
Signed-off-by: Daniel Afrimi <dafrimi@nvidia.com >
Signed-off-by: <dafrimi@nvidia.com >
2026-07-07 11:45:13 +00:00
Harry Mellor and GitHub
0ed05b6f82
[CI] Fix Transformers modeling backend LoRA test ( #47832 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-07-07 11:40:00 +00:00
Guan-Ming Chiu and GitHub
ed051fab54
[Bugfix] Reject sampling params unsupported by diffusion models ( #45418 )
...
Signed-off-by: Guan-Ming (Wesley) Chiu <105915352+guan404ming@users.noreply.github.com >
2026-07-07 11:25:36 +00:00
3354dba381
[Bugfix][KV offload] Store interior chunk-boundary blocks under MTP/Eagle ( #46972 )
...
Signed-off-by: Mikhail Kostryukov <mike@triptrack.net >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Or Ozeri <oro@il.ibm.com >
2026-07-07 13:16:52 +03:00
cbb5f045be
[ROCm][CI] Refresh ROCm base images when docker rocm_base changes ( #46904 )
...
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com >
Signed-off-by: Codex <codex@example.invalid >
Signed-off-by: Micah Williamson <micah.williamson@amd.com >
Signed-off-by: Matthew Wong <Matthew.Wong2@amd.com >
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: Rohan138 <rohanpotdar138@gmail.com >
Co-authored-by: Codex <codex@example.invalid >
Co-authored-by: Micah Williamson <micah.williamson@amd.com >
Co-authored-by: Matthew Wong <Matthew.Wong2@amd.com >
2026-07-07 03:10:50 -07:00
b3e85be663
fix: use configured max_logprobs instead of hardcoded 20 in derender validation ( #47834 )
...
Signed-off-by: jperezde <jperezde@redhat.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-07-07 09:42:47 +00:00
c5b66233b2
[Bugfix][Spec Decode] Skip uniform spec-decode padding for diffusion models ( #47464 )
...
Signed-off-by: kl527 <kl527@cornell.edu >
Signed-off-by: Kyung Sub Lee (Daniel) <66861800+kl527@users.noreply.github.com >
Co-authored-by: Claude Fable 5 <noreply@anthropic.com >
2026-07-07 09:25:12 +00:00
e55cc59e52
[Rust Frontend][CI] Unblock more end-to-end test cases ( #47735 )
...
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
2026-07-07 08:27:20 +00:00
8e61b646e2
fix(security): add resource bounds validation to derender endpoints ( #47260 )
...
Signed-off-by: jperezde <jperezde@redhat.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-07-07 14:58:26 +08:00
e040899a00
[KV Offloading] Add basic offloading metrics ( #45958 )
...
Signed-off-by: srinivas_oo7 <sklinkedin0120@gmail.com >
Signed-off-by: Srinivas Krovvidi <194645829+Srinivasoo7@users.noreply.github.com >
Co-authored-by: srinivas_oo7 <sklinkedin0120@gmail.com >
Co-authored-by: Or Ozeri <or@ozery.com >
Co-authored-by: Or Ozeri <oro@il.ibm.com >
2026-07-07 09:26:28 +03:00
dd5c299fbe
[ROCm][Bugfix] Convert ModelOpt FP8 per-channel weights to e4m3fnuz on MI300/MI325 ( #47201 )
...
Signed-off-by: Micah Williamson <micah.williamson@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-07-06 23:24:58 -07:00
cbe9c40f99
[Bugfix] Forward callable hf_overrides to the draft model config ( #45352 )
...
Signed-off-by: HumphreySun98 <humphreysun98@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-07-06 21:12:38 -07:00
32ab064621
[UX] Add model_class_overrides for development and debugging ( #47148 )
...
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-07-07 11:43:08 +08:00
700e882eab
Add TorchCodec as a video decoding backend ( #46609 )
...
Signed-off-by: Nicolas Hug <contact@nicolas-hug.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn >
2026-07-06 19:58:51 -07:00
Rahul Vishwakarma and GitHub
9dd2465896
feat(cpu): add CPU support for Mamba ShortConv ( #35059 )
...
Signed-off-by: Rahul Vishwakarma <Rahul.Vishwakarma2@ibm.com >
2026-07-07 10:47:08 +08:00
86db6c3070
[Frontend] add per-request timing metrics field to response body of Chat/Completions APIs ( #46768 )
...
Signed-off-by: Nicholas Edelman <nedelman@nvidia.com >
Signed-off-by: Simon Mo <simon@inferact.ai >
Co-authored-by: Simon Mo <simon@inferact.ai >
Co-authored-by: GPT-5.5 <noreply@cursor.com >
2026-07-06 17:48:29 -07:00
5769a7382c
[ROCm][CI][Bugfix] Fix flaky parallel tool-call streaming (test assertion + Mistral/Granite parsers) ( #47550 )
...
Signed-off-by: Aakif Nawaz <aakif.nawaz@amd.com >
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-07-06 19:06:19 -04:00
d891b9bd51
[Quantization] add humming moe backend to all dense/moe oracles ( #41652 )
...
Signed-off-by: Jinzhen Lin <jinzhen.ljz@antgroup.com >
Co-authored-by: mgoin <mgoin64@gmail.com >
2026-07-06 13:36:07 -07:00
Harry Mellor and GitHub
ae098abe3f
[CI] Fix some errors on main ( #47726 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-07-06 19:40:23 +00:00
b1384f5ec6
Enable B12x backend for non-gated MoEs (like Nemotron) ( #43328 )
...
Signed-off-by: Andrii Skliar <askliar@nvidia.com >
Co-authored-by: Andrii Skliar <askliar@nvidia.com >
2026-07-06 12:40:07 -07:00
9fde043f54
[Kernel][Helion][1/N] Add Helion kernel for silu_and_mul_per_block_quant ( #43994 )
...
Signed-off-by: Sean Chen <seachen@redhat.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-07-07 00:19:01 +08:00
5bce653e09
Make the Transformers modeling backend as fast as native vLLM ( #47187 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-07-06 16:59:14 +01:00
598d51153a
[Bugfix][Distributed] Delegate MNNVL allreduce one-shot selection ( #47589 )
...
Signed-off-by: jesco-absolut <team@srswti.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-07-06 07:47:06 -07:00
Harry Mellor and GitHub
51ee564e56
[CI] Skip test for checkpoint that was deleted ( #47748 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-07-06 07:24:09 -07:00
373eb314af
[Bugfix][Core] Fix num_output_placeholders underflow with async scheduling + spec decode ( #46066 )
...
Signed-off-by: Ting Sun <suntcrick@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-07-06 13:50:38 +00:00
07f9baf756
Revert "[Platform] Replace torch.cuda.Event with torch.Event ( #47140 )" ( #47668 )
...
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-07-06 14:18:33 +01:00
BadrBasowid and GitHub
740f379fae
[ROCm][AITER] Directly Implement AITER Custom All-reduce in CudaCommunicator ( #46065 )
...
Signed-off-by: BadrBasowid <badr.basowid@gmail.com >
2026-07-06 12:16:32 +00:00
Alexis K. and GitHub
40cc2e8327
[Bugfix] Return HTTP 422 for unprocessable image URLs instead of 500 ( #47165 )
...
Signed-off-by: Alexis Kinsella <alexis.kinsella@gmail.com >
2026-07-06 11:56:23 +00:00
ba22152096
fix(security): block request-level GPU video backend selection withou… ( #47259 )
...
Signed-off-by: jperezde <jperezde@redhat.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-07-06 02:36:49 -07:00
Andreas Karatzas and GitHub
8f0e75e16b
[ROCm][CI] Adding nixl multiconn ( #47481 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
2026-07-06 15:04:58 +08:00
98ba9b9583
[Frontend] Support OpenAI Responses API namespace tools ( #47024 )
...
Signed-off-by: zhongjing123 <jimzhong5193@gmail.com >
Co-authored-by: zhongjing123 <jimzhong5193@gmail.com >
2026-07-06 06:21:27 +00:00
d9c1767cd4
[INC][ARK] Direct Register Custom Op for ARK ( #46361 )
...
Signed-off-by: Zhenzhong1 <zhenzhong.xu@intel.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-07-06 13:45:50 +08:00
Fadi Arafeh and GitHub
f1073c050c
[CPU][BugFix] Multiple fixes to w4a8_int8 CPU MoE path ( #46739 )
...
Signed-off-by: Fadi Arafeh <fadi.arafeh@arm.com >
2026-07-06 05:39:20 +00:00
Qiming Zhang and GitHub
394edc8108
[XPU] limit max-num-seqs in test_lmeval.py for XPU ( #47682 )
...
Signed-off-by: mauyuyuace <qiming1.zhang@intel.com >
2026-07-06 05:34:16 +00:00
69715823df
[Test][XPU] Skip fork in kv_sharing_fast_prefill test on XPU ( #47406 )
...
Signed-off-by: Ma, Liangliang <liangliang.ma@intel.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-07-06 11:32:26 +08:00
Chaojun Zhang and GitHub
6569df6a3e
[Test][LoRA] Use lightweight CPU reference and skip heavy cleanup in punica ops tests ( #47534 )
...
Signed-off-by: Chaojun Zhang <chaojun.zhang@intel.com >
2026-07-06 11:29:59 +08:00
