f768b4473e
[Docs] Add docs for context extension using the yarn method ( #37430 )
...
Signed-off-by: xiaoming <1259730330@qq.com >
Signed-off-by: labAxiaoming <34019940+labAxiaoming@users.noreply.github.com >
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com >
2026-04-24 08:26:09 -07:00
Jiangyun Zhu and GitHub
e8ee2a78db
[Attention] use diff kv backend for mimo v2 flash ( #40045 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
2026-04-24 11:25:55 +00:00
Dmitry Tokarev and GitHub
6dec49f27e
[Build] Bump CUDA to 13.0.2 to match PyTorch 2.11.0 ( #40669 )
...
Signed-off-by: Dmitry Tokarev <dtokarev@nvidia.com >
2026-04-24 10:27:11 +00:00
079a4cf399
[MoE] Move cutlass moe to fused_moe/experts/ ( #40574 )
...
Signed-off-by: Jackmin801 <ongjackm@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-04-24 06:05:49 +00:00
9744b699ba
[Deprecate] Deprecate LLM.reward offline api, use LLM.encode instead. ( #40688 )
...
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
Signed-off-by: wang.yuqi <noooop@126.com >
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com >
2026-04-24 05:37:50 +00:00
ff2c2bd80a
[Docs]Add documentation for bench serve visualization arguments ( #40539 )
...
Signed-off-by: Sophie du Couédic <sop@zurich.ibm.com >
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2026-04-23 15:48:29 -07:00
0098db9ec1
[ROCm] Implement GPU-to-NUMA-node detection ( #40015 )
...
Signed-off-by: Patrick Schlangen <pschlan@amd.com >
Co-authored-by: TJian <tunjian.tan@embeddedllm.com >
2026-04-23 10:08:48 -05:00
Kunshang Ji and GitHub
53ecc807c0
[XPU] Upgrade torch 2.11 for xpu ( #37947 )
...
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com >
2026-04-23 10:07:35 -05:00
d0009ddb0b
[Model] Support Hy3 preview ( #40681 )
...
Signed-off-by: stevenkuang <stevenkuang@tencent.com >
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com >
2026-04-23 22:08:26 +08:00
Honglin Cao and GitHub
9c271f9403
[gRPC] Add standard gRPC health checking (grpc.health.v1) for Kubernetes native probes ( #38016 )
...
Signed-off-by: Honglin Cao <Caohonglin317@hotmail.com >
2026-04-22 21:31:00 +00:00
ecbe42e991
[Doc] Clarify supported keys for --speculative-config ( #40455 )
...
Signed-off-by: Wangxiaoxiaoa <Wangxiaoxiaoa@users.noreply.github.com >
Co-authored-by: Wangxiaoxiaoa <Wangxiaoxiaoa@users.noreply.github.com >
2026-04-22 04:36:17 -07:00
6aa057c9d7
[Multimodal] Support custom video metadata for pre-extracted frame sequences ( #40133 )
...
Signed-off-by: storyicon <storyicon@foxmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-04-22 15:50:04 +08:00
philip-essential and GitHub
123674879e
[Model] Add block-local attention and YaRN for local layers to Gemma3 ( #39823 )
...
Signed-off-by: Philip Monk <169196560+philip-essential@users.noreply.github.com >
2026-04-21 23:34:50 -07:00
4eafc72928
[Audio] Bundle get_generation_prompt() params into SpeechToTextParams ( #36268 )
...
Signed-off-by: Ekagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com >
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com >
2026-04-22 12:24:18 +08:00
4506319a28
[compile] mla + group fp8 fusion ( #38877 )
...
Signed-off-by: Carl You <4531192+carlyou@users.noreply.github.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-04-21 23:16:58 -04:00
Shanshan Shen and GitHub
936e0b79aa
[MM][CG] Optimize default max_frames_per_batch auto-infer for ViT CUDA graph video inference ( #40445 )
...
Signed-off-by: shen-shanshan <467638484@qq.com >
2026-04-21 14:47:53 +00:00
ec5ef0ac73
[Doc] Add Qwen3 AWQ models to documentation ( #40034 )
...
Signed-off-by: Yusuf <yusufmohammad@live.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-04-21 09:37:41 -04:00
d249a9e90e
Add Granite 4.1 Vision as built-in multimodal model ( #40282 )
...
Signed-off-by: Artem Spector <artems@il.ibm.com >
Signed-off-by: artemspector <artems@il.ibm.com >
Co-authored-by: artemspector <artems@il.ibm.com >
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com >
2026-04-21 05:43:39 -07:00
d2e2e856ad
[Frontend] Remove frontend pooling multi task support. ( #37861 )
...
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
Signed-off-by: wang.yuqi <noooop@126.com >
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-04-21 12:27:44 +00:00
Wentao Ye and GitHub
301024aa9c
[Deprecation] Deprecate cprofile and cprofile_context ( #39100 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-04-21 11:25:22 +08:00
Shanshan Shen and GitHub
8097591286
[Doc] Update ViT CUDA graph doc for mixed (image+video) inputs ( #40355 )
...
Signed-off-by: shen-shanshan <467638484@qq.com >
2026-04-21 02:31:09 +00:00
bai and GitHub
191e3fdaa1
Update flashinfer to 0.6.8 ( #39959 )
...
Signed-off-by: bai <v@gor.io >
2026-04-20 10:37:23 -07:00
Rita Brugarolas and GitHub
fb5635d3f9
[ROCm] Add MLA dual RMS norm fusion (Q, KV) pass for DeepSeek/Kimi-K2 ( #39242 )
...
Signed-off-by: Rita Brugarolas Brufau <rita.brugarolasbrufau@amd.com >
2026-04-20 14:56:27 +00:00
7243e02aa1
[ROCm][Feature] Enable AITER MLA attention backend to work with Eagle3 speculative decoding on ROCm ( #39616 )
...
Signed-off-by: larryli2-amd <larryli2@amd.com >
Co-authored-by: TJian <tunjian.tan@embeddedllm.com >
2026-04-20 09:44:43 -05:00
Vasiliy Kuznetsov and GitHub
38fa87caca
mxfp8 online quant move to new frontend ( #40152 )
...
Signed-off-by: Vasiliy Kuznetsov <vasiliy@meta.com >
2026-04-20 06:26:12 -07:00
Ilya Markov and GitHub
50dd4cb427
[EPLB] Add nixl-based eplb communicator ( #36276 )
...
Signed-off-by: ilmarkov <markovilya197@gmail.com >
Signed-off-by: Markov Ilya <markovilya19@gmail.com >
2026-04-20 10:24:23 +00:00
milesial and GitHub
6d8b80802b
[Docs] Fix thinking_token_budget docs ( #40316 )
...
Signed-off-by: milesial <milesial@users.noreply.github.com >
2026-04-20 08:09:44 +00:00
Lxx and GitHub
d886c26d4d
[Doc] Fix typos in token_embed pooling documentation ( #40266 )
...
Signed-off-by: YifanLi3 <lyfqlx3@gmail.com >
2026-04-19 19:27:32 -07:00
Jee Jee Li and GitHub
bfde49e287
[DOC] Add fuse_minimax_qk_norm ( #39782 )
...
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com >
2026-04-18 00:41:37 -07:00
153ba7f0f3
[Refactor] Drop direct dependency on librosa ( #39079 )
...
Signed-off-by: Nick Cao <ncao@redhat.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-04-18 06:55:38 +00:00
z1ying and GitHub
d0697cc7b6
[Doc] Add Realtime Transcription section to supported_models.md ( #39845 )
...
Signed-off-by: Ziying Tao <tzzying@outlook.com >
2026-04-18 03:26:14 +00:00
Michael Goin and GitHub
1174723eba
Fix TURBOQUANT backend selection in cuda.py ( #40060 )
...
Signed-off-by: Michael Goin <mgoin64@gmail.com >
2026-04-17 07:31:41 -07:00
Ben Browning and GitHub
70770268c3
Add @bbrowning to CODEOWNERS ( #40141 )
...
Signed-off-by: Ben Browning <bbrownin@redhat.com >
2026-04-17 09:51:48 -04:00
wang.yuqi and GitHub
8d2cff8140
[Examples] Resettle Observability examples. ( #40123 )
...
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
2026-04-17 03:13:31 -07:00
z1ying and GitHub
bf45e6d0a5
[Doc] Add Gemma 4 to supported models list ( #39607 )
...
Signed-off-by: z1ying <tzzying@outlook.com >
Signed-off-by: Ziying Tao <tzzying@outlook.com >
2026-04-17 13:42:52 +08:00
Matthew Bonanni and GitHub
219bb5b8c0
[Misc] Update committers.md ( #40058 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
2026-04-16 13:48:41 -07:00
Flora Feng and GitHub
b16fda62b7
[Misc] Add @sfeng33 to CODEOWNERS ( #40048 )
...
Signed-off-by: sfeng33 <4florafeng@gmail.com >
2026-04-16 12:25:29 -07:00
4e8c3f1c19
[Frontend][last/5] Improve pooling entrypoints | clean up. ( #39675 )
...
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
Signed-off-by: wang.yuqi <noooop@126.com >
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2026-04-16 07:53:23 -07:00
Vasiliy Kuznetsov and GitHub
5e5afafa21
[Doc] add docs for online quant frontend ( #39736 )
...
Signed-off-by: Vasiliy Kuznetsov <vasiliy@meta.com >
2026-04-16 07:52:58 -07:00
2cdf86044d
Add Jina Embeddings v5 model support ( fixes #38633 ) ( #39575 )
...
Signed-off-by: Abhijit <abroy@redhat.com >
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
Co-authored-by: wang.yuqi <yuqi.wang@daocloud.io >
2026-04-16 06:37:10 +00:00
R3hankhan and GitHub
4b7ca37bd4
[CPU][IBM Z][Dockefile][Docs] Fix s390x builds for torch 2.11 and update docs for s390x ( #39910 )
...
Signed-off-by: Rehan Khan <Rehan.Khan7@ibm.com >
2026-04-15 22:26:21 -07:00
03f8d3a548
Update to transformers v5 ( #30566 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Signed-off-by: khluu <khluu000@gmail.com >
Signed-off-by: Kevin H. Luu <khluu000@gmail.com >
Signed-off-by: Cyrus Leung <cyrus.tl.leung@gmail.com >
Co-authored-by: khluu <khluu000@gmail.com >
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com >
Co-authored-by: jiang1.li <jiang1.li@intel.com >
2026-04-15 16:29:15 -07:00
3cc328a4be
[SpecDecode][Benchmark] Add SPEED-bench support to benchmarking CLI ( #36029 )
...
Signed-off-by: talora <talora@nvidia.com >
Co-authored-by: Benjamin Chislett <bchislett@nvidia.com >
2026-04-15 12:00:07 -04:00
f4b42df048
[Attention Backend] TurboQuant: 2-bit KV cache compression with 4x capacity ( #38479 )
...
Signed-off-by: vibhavagarwal5 <vibhavagarwal5@gmail.com >
Signed-off-by: Michael Goin <mgoin64@gmail.com >
Co-authored-by: Xinyu Chen <xinyu1.chen@intel.com >
Co-authored-by: Michael Goin <mgoin64@gmail.com >
2026-04-14 19:57:13 -07:00
1a9353bb02
[MoE] Move GPT OSS Triton kernel experts into fused_moe/experts/ ( #39007 )
...
Signed-off-by: Robert Shaw <robertgshaw2@gmail.com >
Signed-off-by: Jackmin801 <ongjackm@gmail.com >
Co-authored-by: Robert Shaw <robertgshaw2@gmail.com >
2026-04-14 19:27:39 +00:00
80118853f4
[MM][Perf][CG] Support ViT full CUDA graph for Qwen3-VL video inference ( #38061 )
...
Signed-off-by: shen-shanshan <467638484@qq.com >
Signed-off-by: Shanshan Shen <87969357+shen-shanshan@users.noreply.github.com >
Co-authored-by: Roger Wang <hey@rogerw.io >
2026-04-14 16:49:32 +08:00
610a3efcaf
[Doc] Fix Python-only build 404 fallback guidance ( #38052 )
...
Signed-off-by: George-ao <yuyiao772@gmail.com >
Signed-off-by: Yuyi Ao <yuyiao772@gmail.com >
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-04-13 12:09:31 -07:00
Yongye Zhu and GitHub
739e5945dc
[Quantization] [Refactor] Create special "GptOssMxfp4MoeMethod" ( #39604 )
...
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
2026-04-13 12:53:58 -04:00
fa6ae31177
feat: rename logit_bias/logit_scale to logit_mean/logit_sigma for affine score calibration ( #39530 )
...
Signed-off-by: Jesus Federico <jefp@amazon.com >
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
Co-authored-by: wang.yuqi <yuqi.wang@daocloud.io >
2026-04-13 04:43:44 +00:00
Nicolò Lucchesi and GitHub
3244a2ebf2
[KVConnector][NIXL] Organize NIXL connector into its own directory ( #39354 )
...
The number of features supported by the connector has grown substantially
and the `nixl_connector.py` file has accumulated a lot of code. Creates a separate
directory and isolates connector/scheduler code in the hope of improving clarity
and maintainability.
Further refactor of components aimed at improving clarity and simplifying code
will follow soon.
Signed-off-by: NickLucche <nlucches@redhat.com >
2026-04-12 13:10:50 +00:00
