khluu
39602ebf8a
[Bugfix] Fix CPU Dockerfile path after cherry-pick
...
Cherry-pick 7bd3f40dda#39557 ) used the upstream requirements/test/
nested layout, but this branch still has the flat requirements/
layout (cpu-test.in not test/cpu.in). Revert the path rename while
keeping the torch==2.11.0 and sentence-transformers==5.3.0 fixes.
Signed-off-by: khluu <khluu000@gmail.com >
2026-04-16 17:46:27 -07:00
Ilya Boytsov and khluu
7225a69c9c
fix(test): recompute Jina ColBERT rotary inv_freq cleared by transformers v5 weight loader ( #39176 )
...
Signed-off-by: Ilya Boytsov <ilyaboytsov1805@gmail.com >
2026-04-16 17:25:30 -07:00
Harry Mellor and khluu
9a3a31fd2d
Don't compile vision encoder for Transformers backend ( #30518 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Signed-off-by: khluu <khluu000@gmail.com >
2026-04-16 17:09:39 -07:00
Li, Jiang and khluu
7bd3f40dda
[CI/Build] Fix sentence-transformers version in CPU test ( #39557 )
...
Signed-off-by: jiang1.li <jiang1.li@intel.com >
(cherry picked from commit eb92ba740akhluu000@gmail.com >
2026-04-16 17:05:39 -07:00
Jee Jee Li and khluu
c5460385f1
[Kernel] Porting the TRTLLM minimax_allreduce_rms kernels ( #37045 )
...
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com >
(cherry picked from commit ecd1ea1363khluu000@gmail.com >
2026-04-16 15:52:31 -07:00
r266-tech and khluu
4bbb8faa1f
fix(kimi_k25): resolve media_placeholder_token_id from tokenizer ( #39344 )
...
Signed-off-by: r266-tech <r266.tech@gmail.com >
Signed-off-by: Roger Wang <hey@rogerw.io >
Co-authored-by: Roger Wang <hey@rogerw.io >
(cherry picked from commit 17e787a779
2026-04-16 15:48:34 -07:00
Harry Mellor and khluu
459d9b38d3
Update to transformers v5 ( #30566 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Signed-off-by: khluu <khluu000@gmail.com >
Signed-off-by: Kevin H. Luu <khluu000@gmail.com >
Signed-off-by: Cyrus Leung <cyrus.tl.leung@gmail.com >
Co-authored-by: khluu <khluu000@gmail.com >
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com >
Co-authored-by: jiang1.li <jiang1.li@intel.com >
(cherry picked from commit 03f8d3a548
2026-04-16 15:47:49 -07:00
Luciano Martins and khluu
b1568cf464
[Model] Fix Gemma 4 token repetition by dynamic BOS injection for PT models ( #39842 )
...
Signed-off-by: Luciano Martins <lucianommartins@users.noreply.github.com >
Co-authored-by: Luciano Martins <lucianommartins@users.noreply.github.com >
(cherry picked from commit 6dc9491406
2026-04-15 17:54:37 -07:00
김의진 and khluu
a4ac72ceba
[Bugfix] Fix Gemma4 tool parser converting bare null to string "null" ( #39679 )
...
Signed-off-by: KimuGenie <baby11686@naver.com >
Signed-off-by: Chauncey <chaunceyjiang@gmail.com >
Co-authored-by: Chauncey <chaunceyjiang@gmail.com >
(cherry picked from commit b075604da1
2026-04-15 17:54:04 -07:00
ShubyM and khluu
0ab0f70aa9
[Gemma4][Bugfix]: Enable Gemma4ForCasualLM to load lora adapters correctly ( #38844 )
...
Signed-off-by: ShubyM <shubymishra20@gmail.com >
(cherry picked from commit 92feb9991d
2026-04-15 17:53:56 -07:00
Fynn Schmitt-Ulms and khluu
ee642f8753
Add Gemma4 Eagle3 support ( #39450 )
...
Signed-off-by: Rahul-Tuli <rtuli@redhat.com >
Signed-off-by: Fynn Schmitt-Ulms <fschmitt@redhat.com >
Co-authored-by: Rahul-Tuli <rtuli@redhat.com >
Co-authored-by: Claude <noreply@anthropic.com >
(cherry picked from commit e7cfd7c5b9
2026-04-15 17:53:46 -07:00
Dipika Sikka and khluu
6db56c0997
[Gemma4] Support quantized MoE ( #39045 )
...
Signed-off-by: Dipika Sikka <dipikasikka1@gmail.com >
(cherry picked from commit 3aecdf08b4
2026-04-15 17:52:46 -07:00
Ben Browning and khluu
9a234c7adc
[Tool] adjust_request to reasoning parser, and Gemma4 fixes ( #39027 )
...
Signed-off-by: Ben Browning <bbrownin@redhat.com >
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
(cherry picked from commit 8477fe427d0d784f4677
2026-04-15 17:51:11 -07:00
Flora Feng and khluu
f56ffafdce
[Bugfix] Fix Gemma4 streaming tool call corruption for split boolean/number values ( #39114 )
...
Signed-off-by: sfeng33 <4florafeng@gmail.com >
(cherry picked from commit 13151a4df4a7b392731f
2026-04-15 17:51:03 -07:00
yoke and khluu
0329e8c9ac
[Bugfix][Frontend] Fix Gemma4 streaming HTML duplication after tool calls ( #38909 )
...
Signed-off-by: yoke233 <yoke2012@gmail.com >
(cherry picked from commit d734445fcdbae948a9a9
2026-04-15 17:50:50 -07:00
Lucas Wilkinson and khluu
6f3dc4d0aa
[Gemma4] Enable Fast Prefill Optimization ( #38879 )
...
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
(cherry picked from commit 47e605092bfc29ef13a0
2026-04-15 17:50:41 -07:00
Greg Pereira and khluu
a6ac49a33b
[Bugfix] Fix invalid JSON in Gemma 4 streaming tool calls by stripping partial delimiters ( #38992 )
...
Signed-off-by: greg pereira <grpereir@redhat.com >
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
(cherry picked from commit f53fa26e0510a26d1d9a
2026-04-15 17:50:31 -07:00
Michael and khluu
2a69949bda
[Bugfix]: Fix Gemma4ToolParser.__init__() missing tools parameter ( #38847 )
...
Signed-off-by: Michael Hospedales <hospedales@me.com >
(cherry picked from commit bb39382b2b
2026-04-02 16:45:38 -07:00
Luciano Martins and khluu
8adcf8c40a
feat(models): implement Google Gemma 4 architecture support (MoE, Multimodal, Reasoning, Tool-Use) ( #38826 )
...
Signed-off-by: Luciano Martins <lucianommartins@users.noreply.github.com >
Signed-off-by: Luciano Martins <lucianomartins@google.com >
Co-authored-by: Luciano Martins <lucianommartins@users.noreply.github.com >
Co-authored-by: Isotr0py <2037008807@qq.com >
(cherry picked from commit 08ed2b9688
2026-04-02 11:49:53 -07:00
khluu
cfad6a509c
Revert "[Bugfix] Restrict TRTLLM attention to SM100, fixing GB300 (SM103) hang ( #38730 )"
...
This reverts commit c284a6671c
2026-04-01 15:14:58 -07:00
Stefano Castagnetta and khluu
c284a6671c
[Bugfix] Restrict TRTLLM attention to SM100, fixing GB300 (SM103) hang ( #38730 )
...
Signed-off-by: Stefano Castagnetta <scastagnetta@nvidia.com >
(cherry picked from commit 6183cae1bd
2026-04-01 12:11:03 -07:00
Chauncey and khluu
3a30a1a6a8
[Misc] Rename think_start_str/think_end_str to reasoning_start_str/reasoning_end_str ( #38242 )
...
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com >
(cherry picked from commit cbe7d18096
2026-04-01 12:10:53 -07:00
Juan Pérez de Algaba and khluu
29982d48b3
(security) Enforce frame limit in VideoMediaIO ( #38636 )
...
Signed-off-by: jperezde <jperezde@redhat.com >
(cherry picked from commit 58ee614221
2026-04-01 12:10:40 -07:00
Yifan Qiao and khluu
1dbbafd3f3
[Feat][v1] Simple yet General CPU KV Cache Offloading ( #37160 )
...
Signed-off-by: Yifan Qiao <yifanqiao@berkeley.edu >
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
(cherry picked from commit 91e4521f9f
2026-04-01 01:03:14 -07:00
Lucas Wilkinson and khluu
0ee3b7fc3d
[Bugfix][MLA] Add logits size budget to sparse indexer prefill chunking ( #36178 )
...
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
(cherry picked from commit eb47454987
2026-04-01 01:02:58 -07:00
Matthew Bonanni and khluu
268bed9cf3
[Bugfix][Async] Fix async spec decoding with hybrid models ( #38556 )
...
Signed-off-by: SandishKumarHN <sandishkumarhn@gmail.com >
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: SandishKumarHN <sandishkumarhn@gmail.com >
(cherry picked from commit 757068dc65
2026-04-01 01:02:35 -07:00
Jiangyun Zhu and khluu
bcc0fdd0f3
[CI] fix LM Eval Qwen3.5 Models (B200) ( #38632 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
(cherry picked from commit ea7bfde6e4
2026-04-01 01:02:20 -07:00
wang.yuqi and khluu
69b8bd4b33
[CI Failure] pin colmodernvbert revision ( #38612 )
...
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
Signed-off-by: wang.yuqi <noooop@126.com >
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
(cherry picked from commit 719735d6c5
2026-04-01 01:02:04 -07:00
Li, Jiang and khluu
12449f9492
[Bugfix][CPU] Skip set_num_threads after thread binding ( #38535 )
...
Signed-off-by: jiang1.li <jiang1.li@intel.com >
(cherry picked from commit 6557f4937f
2026-03-30 23:01:42 -07:00
haosdent and khluu
b92312dfd7
[CI] Fix SPLADE pooler test broken by #38139 ( #38495 )
...
Signed-off-by: haosdent <haosdent@gmail.com >
(cherry picked from commit a08b7733fd
2026-03-30 21:52:13 -07:00
Jaewon and GitHub
d816834c1a
[MoE] Add RoutingMethodType.Simulated to TRT-LLM FP8/NVFP4 kernel allowlists ( #38329 )
...
Signed-off-by: Jaewon Lee <jaewon@meta.com >
2026-03-29 22:53:43 -07:00
Roger Wang and GitHub
92f0db57a8
[Misc] Always use forward_mulmat for Conv3d on newer versions of torch. ( #38487 )
2026-03-30 05:39:41 +00:00
Andreas Karatzas and GitHub
bea23536f6
[CI] Add temperature=0.0, reduce max_tokens, and add debug prints to audio_in_video tests ( #38492 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
2026-03-30 05:36:45 +00:00
Jiangyun Zhu and GitHub
c133f33746
Add @ZJY0516 to CODEOWNERS ( #38497 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
2026-03-29 21:10:00 -07:00
a6db99ba02
[Bugfix] Support multi-type params parsing for DeepSeek v3.2 ( #33703 )
...
Signed-off-by: Stanislav Kirillov <stas@nebius.com >
Co-authored-by: Stanislav Kirillov <stas@nebius.com >
Co-authored-by: Chauncey <chaunceyjiang@gmail.com >
2026-03-30 04:07:28 +00:00
Andreas Karatzas and GitHub
4f2ed5fddb
[ROCm][CI] Enable hybrid chunked prefill test ( #38317 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
2026-03-30 10:30:26 +08:00
Kyle Sayers and GitHub
d28d86e8a3
[QeRL] Fix online quantized reloading ( #38442 )
...
Signed-off-by: Kyle Sayers <kylesayrs@gmail.com >
2026-03-29 14:56:41 -06:00
Wentao Ye and GitHub
995dea1354
[Perf] Remove redundant device copies for CPU-only pooling token IDs, 48.9% E2E throughput improvement ( #38139 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-03-29 18:12:50 +00:00
allgather and GitHub
8c0b6267d7
[Transformers v5] fix missing pixtral/voxtral multimodal dispatch ( #38410 )
...
Signed-off-by: allgather <all2allops@gmail.com >
2026-03-29 09:59:06 +00:00
Andreas Karatzas and GitHub
43cc5138e5
[ROCm][CI] Fix cross-attention dispatch for encoder-decoder models ( #38450 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
2026-03-28 22:08:03 -07:00
Shubhra Pandit and GitHub
5b8c30d62b
[Spec Decode, BugFix] Propagate norm_before_fc from Eagle3 speculator ( #38111 )
...
Signed-off-by: Shubhra Pandit <shubhra.pandit@gmail.com >
2026-03-29 00:42:06 +00:00
haosdent and GitHub
d39b8daf5f
[Feature] Add Qwen3-ForcedAligner support via token classification pooling ( #35367 )
...
Signed-off-by: haosdent <haosdent@gmail.com >
2026-03-29 00:27:52 +00:00
Walter Beller-Morales and GitHub
fafca38adc
[BugFix][Frontend] apply task instruction as system prompt in cohere v2/embed ( #38362 )
...
Signed-off-by: walterbm <walter.beller.morales@gmail.com >
2026-03-28 18:30:54 +00:00
aa4eb0db78
[CI]revert initialize_model context manager ( #38426 )
...
Signed-off-by: Kunshang Ji <jikunshang95@gmail.com >
Co-authored-by: wang.yuqi <yuqi.wang@daocloud.io >
2026-03-28 16:56:50 +00:00
Andreas Karatzas and GitHub
af89140efc
[ROCm][CI] Fix UV install in Dockerfile.rocm to detect curl failures and retry ( #38415 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
2026-03-29 00:47:42 +08:00
haosdent and GitHub
b2bc736b12
[CI] Fix Ernie4.5-VL initialization test ( #38429 )
...
Signed-off-by: haosdent <haosdent@gmail.com >
2026-03-28 22:43:24 +08:00
whyiug and GitHub
58c959a767
[Misc]: clean up non-core lint issues ( #37049 )
...
Signed-off-by: whyiug <whyiug@hotmail.com >
2026-03-28 10:28:16 -04:00
Bvicii and GitHub
bda3eda82d
[Bugfix] Disallow renderer_num_workers > 1 with mm processor cache ( #38418 )
...
Signed-off-by: Bvicii <yizhanhuang2002@gmail.com >
2026-03-28 06:32:52 -07:00
Michael Goin and GitHub
2bf5b70ae8
[CI Bugfix] Pre-download missing FlashInfer headers in Docker build ( #38391 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
2026-03-28 06:09:00 -07:00
yzong-rh and GitHub
6dad4c5722
[Test] Fix flaky race condition in test_abort_final_step ( #38414 )
...
Signed-off-by: Yifan <yzong@redhat.com >
2026-03-28 09:06:56 +00:00
khluu