|
bf18d7e0b4
|
[Misc][NUMA] Auto-bind to PCT priority cores on DGX B300 + widen EngineCore across shard NUMA nodes (#43270)
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>
Co-authored-by: Cursor <noreply@cursor.com>
|
2026-05-29 10:07:44 +08:00 |
|
|
|
b690b2bb67
|
[Model]Support Step-3.7-Flash (#43859)
Signed-off-by: luotingdan <luotingdan@stepfun.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: luotingdan <luotingdan@stepfun.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Yu Huang <yuhuang@nvidia.com>
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai>
|
2026-05-28 17:01:48 -07:00 |
|
|
|
69c9f19957
|
fix(frontend): Add multimodal placeholders to Gemma4 tool message template (#41459)
Signed-off-by: Harshal Janjani <harshaljanjani@gmail.com>
Co-authored-by: Ben Browning <bbrownin@redhat.com>
|
2026-05-28 14:48:12 -07:00 |
|
|
rasmithandGitHub
|
9769e2df2a
|
[AMD][CI][BugFix] Fix Distributed Compile Unit Tests (2xH100-2xMI300) group (#43120)
Signed-off-by: Randall Smith <Randall.Smith@amd.com>
|
2026-05-28 14:39:01 -07:00 |
|
|
|
a3ed5ab10c
|
[KV Offload] Add per-request offloading policy via on_new_request lifecycle hook (#43205)
Signed-off-by: Ronen Schaffer <ronen.schaffer@ibm.com>
Co-authored-by: Or Ozeri <or@ozery.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-05-28 20:45:18 +00:00 |
|
|
|
7e53283b1c
|
[Core] Cleanup KVConnector handling with PP + fix MRV2 (#43732)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-05-28 13:12:03 -07:00 |
|
|
|
9090368b65
|
[Feat] Add support for per GPU worker RDMA NIC selection (#42083)
Signed-off-by: Raj Joshi <rajjoshi@redhat.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
|
2026-05-28 12:45:23 -07:00 |
|
|
Harry MellorandGitHub
|
085ac221a3
|
Deprecate JAISLMHeadModel (#43784)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
|
2026-05-28 18:29:12 +00:00 |
|
|
|
9aa131f944
|
Add Cosmos3 Reasoner model (#43356)
Signed-off-by: Maciej Bala <mbala@nvidia.com>
Signed-off-by: MaciejBalaNV <mbala@nvidia.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Isotr0py <2037008807@qq.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
|
2026-05-28 09:43:55 -07:00 |
|
|
Ronen SchafferandGitHub
|
4bfa0f2b14
|
[KV Offload] Rename SecondaryTierManager.get_finished() to get_finished_jobs() (#43870)
Signed-off-by: Ronen Schaffer <ronen.schaffer@ibm.com>
|
2026-05-28 16:00:18 +00:00 |
|
|
Vadim GimpelsonandGitHub
|
5d126dd155
|
[Bugfix] Exclude Ray DP from #42585's deferred port allocation (#43864)
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>
|
2026-05-28 15:55:14 +00:00 |
|
|
|
864990e8d9
|
Add token-offset based selective offload in OffloadConnector (#39983)
Signed-off-by: Angelo Ruocco <ang@zurich.ibm.com>
Co-authored-by: Or Ozeri <or@ozery.com>
|
2026-05-28 14:11:02 +00:00 |
|
|
|
f3b2a819f7
|
[Perf][KDA] Fuse gate softplus, chunk-local cumsum, and RCP_LN2 scaling (#43667)
Signed-off-by: haojiangzheng <justineric096@gmail.com>
Co-authored-by: haojiangzheng <justineric096@gmail.com>
|
2026-05-28 13:47:08 +00:00 |
|
|
Wentao YeandGitHub
|
64e1218673
|
[Perf] Optimize moe permute by pre-allocate buffer, 9~14% kernel performance improvement (#43014)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
|
2026-05-28 06:18:26 -07:00 |
|
|
ChaunceyandGitHub
|
d692b89c2c
|
[Feature] Add structured output and effort support to Anthropic Messages API (#42396)
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
|
2026-05-28 12:06:48 +00:00 |
|
|
|
61288b5458
|
[Bugfix] Fix HyperCLOVAX CI failure after upstream removed remote code (#43860)
Signed-off-by: Kevin Luu <kevin@inferact.ai>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
2026-05-28 03:37:36 -07:00 |
|
|
|
b372ad3e90
|
[Bugfix] Stream DeepSeek DSML tool-call argument deltas incrementally (#42879)
Signed-off-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com>
Co-authored-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
|
2026-05-28 17:50:23 +08:00 |
|
|
Rotem ShavittandGitHub
|
1b16f2ddc9
|
change name of fs_python secondary tier to fs. (#43600)
Signed-off-by: Rotem Shavitt <rshavitt@gmail.com>
|
2026-05-28 07:05:48 +00:00 |
|
|
TJianandGitHub
|
0ba46d4b11
|
[ROCm][DSV4] Enable Tilelang MHC replacing torch/triton mhc (#43679)
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
|
2026-05-28 07:05:28 +00:00 |
|
|
|
7909f82a45
|
[Bugfix][Frontend] streaming tool-call serializer drops first args chunk when name and args share a DeltaMessage (#42683)
Signed-off-by: ignaciosica <mignacio.sica@gmail.com>
Signed-off-by: sfeng33 <4florafeng@gmail.com>
Co-authored-by: sfeng33 <4florafeng@gmail.com>
|
2026-05-28 05:20:55 +00:00 |
|
|
Nick HillandGitHub
|
626fa9bba5
|
[BugFix] Fix blocked reasoning parsing with MRV2 (#43808)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
|
2026-05-28 04:59:34 +00:00 |
|
|
Thien TranandGitHub
|
e54eff769d
|
[Bugfix] Pass routed_scaling_factor to FlashInfer TRTLLM BF16 MoE (#43769)
|
2026-05-27 21:29:14 -07:00 |
|
|
|
05ac829629
|
fix: parse Qwen3 XML JSON arguments first (#43243)
Signed-off-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
|
2026-05-28 03:35:59 +00:00 |
|
|
Yongye ZhuandGitHub
|
2d2c660104
|
[MoE] Remove inplace fused experts mechanism (#43727)
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
|
2026-05-27 20:00:19 -07:00 |
|
|
Benjamin BartelsandGitHub
|
05eec7120e
|
Fix RunAI streamer tensor buffer reuse during weight loading (#43464)
Signed-off-by: bbartels <benjamin@bartels.dev>
|
2026-05-27 19:16:52 -07:00 |
|
|
jatseng-aiandGitHub
|
05c50c721e
|
[ROCm] mori: add InterNodeV1LL inter-node kernel selection via VLLM_MORI_INTERNODE_KERNEL (#41751)
Signed-off-by: jatseng-ai <jatseng@amd.com>
|
2026-05-28 00:33:32 +08:00 |
|
|
Injae RyouandGitHub
|
165460941f
|
[BugFix] HFValidationError with cloud storage URIs when HF_HUB_OFFLINE=1 (#39155)
Signed-off-by: Injae Ryou <injaeryou@gmail.com>
|
2026-05-27 10:53:32 -05:00 |
|
|
|
52a31ccecc
|
[Bugfix] Map reasoning_effort to enable_thinking in chat template kwargs (#43401)
Signed-off-by: Ashwin Giridharan <girida@amazon.com>
Signed-off-by: Chauncey <chaunceyjiang@gmail.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
|
2026-05-27 05:39:49 -07:00 |
|
|
|
683033d4ba
|
[Frontend] Add MiniCPM5 XML tool call parser (#43175)
Signed-off-by: zhangtao <zhangtao2@modelbest.cn>
Signed-off-by: zhangtao2 <zhangtao2@modelbest.cn>
Co-authored-by: zhangtao <zhangtao2@modelbest.cn>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
|
2026-05-27 00:39:35 -07:00 |
|
|
|
8c94938cfb
|
[MRV2][BugFix] Fix KV connector handling in spec decode case (#43719)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
|
2026-05-27 06:37:56 +00:00 |
|
|
|
1fc2cee50a
|
[KVConnector][Mooncake] Wire reset_cache cascade end-to-end (#42694)
Signed-off-by: aoshen524 <aoshen524@gmail.com>
Signed-off-by: Ao Shen <aoshen@inferact.ai>
Co-authored-by: aoshen524 <aoshen524@gmail.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
2026-05-26 20:52:35 -07:00 |
|
|
Angela YiandGitHub
|
0fa3114ae1
|
Fix test_aot_compile for torch 2.12 (#43695)
Signed-off-by: Angela Yi <yiangela7@gmail.com>
|
2026-05-26 23:12:49 -04:00 |
|
|
Matthew BonanniandGitHub
|
aa6138169f
|
[MLA][Attention] Add OOT MLA prefill backend registration mechanism (#43325)
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
|
2026-05-26 19:56:09 -07:00 |
|
|
|
7e33081cee
|
[Attention] Make FlexAttention and FlashAttention use num-blocks first layouts (#42095)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com>
|
2026-05-26 19:55:56 -07:00 |
|
|
|
812e7e7364
|
[Bugfix][V1] Fix TOCTOU race causing intermittent EADDRINUSE on multi-API-server DP startup (#42585)
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>
Signed-off-by: Vadim Gimpelson <156319763+vadiklyutiy@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
|
2026-05-26 14:06:00 -07:00 |
|
|
Jee Jee LiandGitHub
|
6e503868ca
|
[Kernel] Porting fuse_minimax_qk_norm to manual fusion (#43410)
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
|
2026-05-26 13:16:03 -07:00 |
|
|
Wei-Ming ChenandGitHub
|
6f5b533241
|
Add LM head quantization support for ModelOpt (#42124)
Signed-off-by: weimingc <17592131+meenchen@users.noreply.github.com>
|
2026-05-26 09:21:05 -07:00 |
|
|
|
f51bbc694d
|
[MoE Refactor] W4a8 int8 oracle (#42789)
Signed-off-by: Bill Nell <bnell@redhat.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
|
2026-05-26 11:15:42 -04:00 |
|
|
Yongye ZhuandGitHub
|
6ab6ffb428
|
[Feat][DSV4] Fuse q pad into deepseek v4 fused kernel (#43162)
|
2026-05-26 05:12:54 -10:00 |
|
|
Andreas KaratzasandGitHub
|
445ded18c1
|
[ROCm][CI] Extend ROCm quick reduce coverage (#40990)
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
|
2026-05-26 21:57:13 +08:00 |
|
|
|
739af5c7e1
|
[Reasoning] [Bugfix] Reject invalid thinking_token_budget values (#43402)
Signed-off-by: linzm1007 <linzm1007@126.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-05-26 03:37:30 -07:00 |
|
|
Thibault CastellsandGitHub
|
5d09f471f4
|
[Misc] Support interleaved custom image benchmark datasets (#43636)
Signed-off-by: ThibaultCastells <thib.castells@icloud.com>
|
2026-05-26 03:37:25 -07:00 |
|
|
|
681d7dd38b
|
[Misc][Refactor][ROCm] Convert MoRI-related envvars to extra config args (#43303)
Signed-off-by: simondanielsson <simon.danielsson99@hotmail.com>
Co-authored-by: TJian <tunjian.tan@embeddedllm.com>
|
2026-05-26 03:33:35 -07:00 |
|
|
Ethan FengandGitHub
|
755043cf3c
|
[KV Transfer] Enable HMA by default for connectors that support it (#41847)
Signed-off-by: Ethan Feng <ethan.fengch@gmail.com>
|
2026-05-26 12:28:51 +02:00 |
|
|
Hank_andGitHub
|
b3269454b1
|
[chores][log] change registry log from warning to debug (#43045)
Signed-off-by: Hank <hcc.mayday@gmail.com>
|
2026-05-26 00:13:46 -07:00 |
|
|
Thien TranandGitHub
|
d56612c621
|
[GDN] GDN Prefill kernel for SM100 (#43273)
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
|
2026-05-26 14:02:11 +08:00 |
|
|
|
d5cf7b4a2c
|
[Frontend] Split the offline inference APIs and utils. (#43553)
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
|
2026-05-26 05:20:24 +00:00 |
|
|
Dao007foreverandGitHub
|
c2a4005c70
|
[KV Connector] Propagate MooncakeStore load failures (#42788)
Signed-off-by: Dao Le <Dao007forever@gmail.com>
|
2026-05-25 22:12:15 -07:00 |
|
|
|
7966fc7233
|
[KV Connector][Bugfix] MooncakeStore: don't double-apply Eagle prune in load_mask (#43516)
Signed-off-by: Dao Le <daole@inferact.ai>
Signed-off-by: Dao Le <Dao007forever@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-05-25 22:11:57 -07:00 |
|
|
Woosuk KwonandGitHub
|
aa2b56ffb0
|
[DeepSeek V4] Move MegaMoE input prep kernel to nvidia/ops (#43632)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
|
2026-05-25 21:08:29 -07:00 |
|