wanglinian
0bbbd388de
perf(chat_utils): use .media instead of .original_bytes to avoid redundant bytes→PIL conversion
...
When processing vision_chunk items, pass the already-decoded PIL.Image
via .media instead of .original_bytes. This avoids an unnecessary
bytes→PIL.Image conversion in media_processor since the image was
already decoded in load_bytes().
Signed-off-by: wanglinian <wanglinian@stu.pku.edu.cn >
2026-01-08 06:06:40 +00:00
wanglinian
c2bfb1def1
docs: Update Kimi-K2.5 model name and remove video support marker
...
Signed-off-by: wanglinian <wanglinian@stu.pku.edu.cn >
2026-01-07 08:59:10 +00:00
wanglinian
5f54da5e7e
Rename K2VL to KimiK25
...
- Rename files: k2vl.py -> kimi_k25.py, k2vl_vit.py -> kimi_k25_vit.py
- Rename classes: K2VLConfig -> KimiK25Config, K2VLForConditionalGeneration -> KimiK25ForConditionalGeneration, etc.
- Update registry, configs/__init__.py, config.py mappings
- Update docs and comments to reference Kimi-K2.5 instead of K2VL
- Pass vision_config to KimiK25MultiModalProjector per reference diff
Signed-off-by: wanglinian <wanglinian@stu.pku.edu.cn >
2026-01-07 08:53:22 +00:00
wanglinian
875d970dd0
refactor(k2vl): remove unused video-chunk config params
...
Remove temporal_merge_kernel_size, sample_fps, and timestamp_mode
from K2VLConfig as they are not used by vLLM.
These parameters are loaded from preprocessor_config.json via
media_processor and should not be duplicated in the model config.
Signed-off-by: wanglinian <wanglinian@stu.pku.edu.cn >
2026-01-07 07:15:54 +00:00
wanglinian
a0a0fb6075
[Model] Refactor K2VL ViT to use MMEncoderAttention
...
- Replace custom multihead_attention/eager_attention with MMEncoderAttention
- Add tensor parallel support using QKVParallelLinear and RowParallelLinear
- MLP2 now uses ColumnParallelLinear/RowParallelLinear with TP/DP support
- Pass multimodal_config through MoonViT3dPretrainedModel to encoder layers
- Rename vision_tower_forward_auto to vision_tower_forward
This aligns with PR #31738 pattern for unified multi-platform attention backends.
Signed-off-by: wanglinian <wanglinian@stu.pku.edu.cn >
2026-01-07 03:36:23 +00:00
wanglinian
bd5bb72eb0
docs: add K2VL model to registry and supported models
...
Signed-off-by: wanglinian <wanglinian@stu.pku.edu.cn >
2026-01-06 05:13:15 +00:00
wanglinian
ffb3ecf3b8
refactor: address code review feedback
...
- k2vl_vit.py: Use get_act_fn('gelu_pytorch_tanh') instead of PytorchGELUTanh
- k2vl_vit.py: Remove VisionTowerConfig and ProjectorConfig wrapper classes
- k2vl.py: Use TensorSchema (K2VLMediaPixelInputs) instead of BatchFeature
- chat_utils.py: Add assert isinstance(item, dict) for type narrowing
Signed-off-by: wanglinian <wanglinian@stu.pku.edu.cn >
2026-01-06 03:58:35 +00:00
wanglinian
dabbad7e04
Address code review feedback: refactor tpool_patch_merger and fix imports
...
Signed-off-by: wanglinian <wanglinian@stu.pku.edu.cn >
2026-01-05 11:41:01 +00:00
wanglinian
a6db17b442
fix: rename mm_projection_auto to mm_projector_forward
...
Signed-off-by: wanglinian <wanglinian@stu.pku.edu.cn >
2026-01-05 05:47:11 +00:00
wanglinian
256bc095ad
refactor: split K2VLConfig vision params into K2VLVisionConfig
...
Signed-off-by: wanglinian <wanglinian@stu.pku.edu.cn >
2026-01-05 04:20:54 +00:00
4502ffce65
Update vllm/entrypoints/chat_utils.py
...
Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com >
Signed-off-by: wangln19 <96399074+wangln19@users.noreply.github.com >
2026-01-05 11:42:23 +08:00
26898aa7aa
Update vllm/entrypoints/chat_utils.py
...
Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com >
Signed-off-by: wangln19 <96399074+wangln19@users.noreply.github.com >
2026-01-05 11:42:02 +08:00
f28234d4bf
Update vllm/model_executor/models/k2vl_vit.py
...
Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com >
Signed-off-by: wangln19 <96399074+wangln19@users.noreply.github.com >
2026-01-05 11:41:51 +08:00
c9243a0482
Update vllm/model_executor/models/k2vl_vit.py
...
Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com >
Signed-off-by: wangln19 <96399074+wangln19@users.noreply.github.com >
2026-01-05 11:41:29 +08:00
wanglinian
120e059650
delete enable_thinking and training para
...
Signed-off-by: wanglinian <wanglinian@stu.pku.edu.cn >
2026-01-05 03:37:40 +00:00
wanglinian
c9d5ff0405
feat: support K2.5 model
...
Add support for K2.5 model (K2VL) in vllm.
Key changes:
- Add K2VL model implementation and configuration.
- Add Kimi K2 reasoning parser.
- Update Chat API and multimodal inputs to support video chunks.
- Add K2.5 specific multimodal processing logic and parsing.
- Register K2VL model and configuration.
Signed-off-by: wanglinian <wanglinian@stu.pku.edu.cn >
2026-01-04 07:25:03 +00:00
Li, Jiang and GitHub
17347daaa2
[CI/Build][CPU] Update CPU CI test cases ( #31466 )
...
Signed-off-by: jiang1.li <jiang1.li@intel.com >
2025-12-29 14:17:52 +08:00
Mamy Ratsimbazafy and GitHub
b9793e6a8c
Add Fused MoE Triton kernels for GLM-4.5-Air, GLM-4.5v, GLM-4.6v on 2x RTX Pro 6000 ( #31407 )
...
Signed-off-by: Mamy Ratsimbazafy <mamy_github@numforge.co >
2025-12-28 08:38:33 -08:00
Jzz1943 and GitHub
0b6b701050
[Model] Add tuned triton fused_moe configs for Qwen3Moe on B200 ( #31448 )
...
Signed-off-by: Zhongze Jiang <jiangzhongze.jzz@ant-intl.com >
2025-12-28 08:38:07 -08:00
Nick Hill and GitHub
094fcce250
[BugFix] Re-fix async multimodal cpu tensor race condition ( #31373 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Signed-off-by: njhill <nickhill123@gmail.com >
2025-12-28 03:05:08 -08:00
Andreas Karatzas and GitHub
573dd0e6f0
[ROCm] Migrate xgrammar to upstream release ( #31327 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
2025-12-28 00:08:29 -08:00
Andreas Karatzas and GitHub
f70368867e
[ROCm][CI] Add TorchCodec source build for transcription tests ( #31323 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
2025-12-28 16:06:05 +08:00
Andreas Karatzas and GitHub
96142f2094
[ROCm][CI] Added perceptron lib in requirements for isaac multi-modal test ( #31441 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
2025-12-28 04:15:14 +00:00
Boyuan Feng and GitHub
62def07d67
[BugFix] register quant scale tensors as buffer ( #31395 )
...
Signed-off-by: Boyuan Feng <boyuan@meta.com >
2025-12-28 11:20:02 +08:00
yitingdc and GitHub
b326598e97
add tip for VLLM_USE_PRECOMPILED arg to reduce docker build time ( #31385 )
...
Signed-off-by: yiting.jiang <yiting.jiang@daocloud.io >
2025-12-28 03:19:47 +00:00
727c41f3fd
[MoE Refactor][10/N] Cleanup Fp8 Process Weights After Loading ( #31169 )
...
Signed-off-by: Robert Shaw <robshaw@redhat.com >
Co-authored-by: Robert Shaw <robshaw@redhat.com >
2025-12-27 20:22:48 +00:00
Boyuan Feng and GitHub
2f12cd32c0
[BugFix] Fix cache issue in compilation_config ( #31376 )
...
Signed-off-by: Boyuan Feng <boyuan@meta.com >
2025-12-27 09:30:39 -05:00
Isotr0py and GitHub
40a8756224
[Chore]: Remove HF format Phi4-MM examples ( #31405 )
...
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn >
2025-12-27 13:42:02 +00:00
Isotr0py and GitHub
3d024985ab
[CI/Build] Ignore max transformers version for more common tests ( #31401 )
...
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn >
2025-12-27 13:06:26 +00:00
8711b21676
Fix/get raw stream patch #30905 ( #30912 )
...
Signed-off-by: baonudesifeizhai <baonudesifeizhai@gmail.com >
Signed-off-by: baonudesifeizhai <85092850+baonudesifeizhai@users.noreply.github.com >
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com >
2025-12-26 20:08:47 -08:00
52bf066516
[Core][Hybrid allocator + connector] Support hybrid allocator + kv cache connector ( #30166 )
...
Signed-off-by: Yifan Qiao <yifanqiao@berkeley.edu >
Co-authored-by: KuntaiDu <kuntai@uchicago.edu >
2025-12-26 18:25:46 -08:00
Kunshang Ji and GitHub
5326c89803
[XPU][CI]skip test_preprocess_error_handling due to fork/spawn issue ( #31381 )
...
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com >
2025-12-26 21:40:44 +00:00
Xinyu Chen and GitHub
87f1b8ca2c
CustomOp: Unify aiter impl into GroupedTopk ( #31221 )
...
Signed-off-by: Xinyu Chen <xinyu1.chen@intel.com >
2025-12-26 12:44:29 -05:00
rongfu.leng and GitHub
887e900b77
[Docs] Add profiler user docs for http request ( #31370 )
...
Signed-off-by: rongfu.leng <rongfu.leng@daocloud.io >
2025-12-26 23:48:15 +08:00
48e744976c
[Mistral common] Ensure all functions are imported from the top & only use public methods ( #31138 )
...
Signed-off-by: Patrick von Platen <patrick.v.platen@gmail.com >
Co-authored-by: Julien Denize <40604584+juliendenize@users.noreply.github.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2025-12-26 04:48:24 -08:00
ce1eafd1a5
[Core] Initialize LoRA support for tower and connector in multi-modal models ( #26674 )
...
Signed-off-by: bk-201 <joy25810@foxmail.com >
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com >
Signed-off-by: prashanth058 <prashanth.dannamaneni@uipath.com >
Co-authored-by: bk-201 <joy25810@foxmail.com >
Co-authored-by: prashanth058 <prashanth.dannamaneni@uipath.com >
Co-authored-by: Anexdeus <5142168@mail.ru >
2025-12-26 04:48:20 -08:00
Harry Mellor and GitHub
0b544e6476
[Docs] Fix some snippets ( #31378 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2025-12-26 12:47:41 +00:00
Jee Jee Li and GitHub
c3666f56fd
[Misc] Fix Qwen2-MoE shared_expert_gate ( #31339 )
...
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com >
2025-12-26 05:10:39 +00:00
Andreas Karatzas and GitHub
c79dbfa9ad
[CI] Fix flaky vision beam search test with flexible semantic validation ( #31324 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
2025-12-26 04:39:32 +00:00
Shinichi Hemmi and GitHub
9ee05cbe7f
Support LoRA and GPTQModel for PLaMo 2/3 ( #31322 )
...
Signed-off-by: Shinichi Hemmi <50256998+Alnusjaponica@users.noreply.github.com >
2025-12-26 11:41:33 +08:00
Ning Xie and GitHub
3b8f31b362
[benchmark] use model card root instead of id ( #31329 )
...
Signed-off-by: Andy Xie <andy.xning@gmail.com >
2025-12-26 10:55:56 +08:00
Isotr0py and GitHub
2cd94259c8
[CI/Build] Ignore max transformers version skipping for initialization tests ( #30619 )
...
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn >
2025-12-26 10:50:32 +08:00
b7165d53c6
Feature/isaac 0.1 ( #28367 )
...
Signed-off-by: oscardev256 <42308241+oscardev256@users.noreply.github.com >
Signed-off-by: Oscar Gonzalez <ogonzal6@alumni.jh.edu >
Signed-off-by: Yang <lymailforjob@gmail.com >
Co-authored-by: Yang <lymailforjob@gmail.com >
2025-12-25 18:49:11 -08:00
Nick Hill and GitHub
81786c8774
[BugFix] Fix async scheduling + reasoning with struct output ( #31332 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
2025-12-25 23:01:02 +00:00
f1531d9f2a
[Hybrid] Mamba2 prefix cache blocks freeing for running requests ( #28047 )
...
Signed-off-by: Stanislaw Wozniak <stw@zurich.ibm.com >
Signed-off-by: Chen Zhang <zhangch99@outlook.com >
Co-authored-by: Chen Zhang <zhangch99@outlook.com >
2025-12-25 20:54:06 +00:00
2d6001f491
[Model][Ernie4.5-VL] Support video metadata for timestamp rendering ( #31274 )
...
Signed-off-by: dengsonghe <dengsonghe@baidu.com >
Co-authored-by: dengsonghe <dengsonghe@baidu.com >
2025-12-25 14:07:15 +00:00
030fc44914
use the same stream for cuda graph catpure and replay for NCCL ( #29207 )
...
Signed-off-by: Amir Samani <asamani@nvidia.com >
Signed-off-by: youkaichao <youkaichao@gmail.com >
Co-authored-by: youkaichao <youkaichao@gmail.com >
2025-12-25 19:10:03 +08:00
2532f437ee
[Doc] Add troubleshooting for Triton PTX error about undefined gpu-name ( #31338 )
...
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn >
Signed-off-by: Isotr0py <2037008807@qq.com >
Co-authored-by: youkaichao <youkaichao@gmail.com >
2025-12-25 02:26:34 -08:00
Louie Tsai and GitHub
f15185fbdb
[Benchmark Suite] improve cpu Benchmark Suite tests and comparison report for 0.12.0 ( #30994 )
...
Signed-off-by: Tsai, Louie <louie.tsai@intel.com >
2025-12-25 08:51:45 +00:00
Mark Gatere and GitHub
ba25a65992
[Frontend] add FunctionGemma tool parser support ( #31218 )
...
Signed-off-by: gateremark <gateremg@gmail.com >
2025-12-25 15:29:25 +08:00
wanglinian