3 people bea02cdf93 Fix routed experts capture for hybrid models (Mamba + Attention) (#35744) ...

Signed-off-by: arlenxu <arlenxu@tencent.com>
Signed-off-by: xhx1022 <1737006628@qq.com>
Co-authored-by: arlenxu <arlenxu@tencent.com>

2026-03-11 08:53:10 -07:00

..

disaggregated-prefill-v1

kv_transfer: Rename the shared storage connectors (#30201)

2025-12-08 20:46:09 -08:00

kv_load_failure_recovery

[PD] Change kv_load_failure_policy Default from "recompute" to "fail" (#34896)

2026-02-21 01:34:57 -08:00

logits_processor

docs: add README for logits_processor examples (#35933)

2026-03-04 17:09:19 +00:00

new_weight_syncing

[ROCm][CI] Support async weight transfer example with platform-aware determinism (#35710)

2026-03-04 09:44:14 +08:00

openai_batch

[Docs] Improve documentation (#33799)

2026-02-06 12:57:09 +00:00

qwen2_5_omni

[Model] Use mm_position to compute mrope positions for Qwen2.5-Omni (#32772)

2026-01-25 20:15:53 +08:00

qwen3_omni

[Model] Use mm_position to compute mrope positions for Qwen3-Omni (#33010)

2026-01-26 13:48:07 +00:00

async_llm_streaming.py

[Example] Add async_llm_streaming.py example for AsyncLLM streaming in python (#21763)

2025-07-30 18:39:46 -06:00

audio_language.py

[Model] Add support for moonshotai/Kimi-Audio-7B-Instruct (#36127)

2026-03-10 21:24:48 -07:00

automatic_prefix_caching.py

Add flake8-implicit-str-concat rules to Ruff (#33191)

2026-01-28 04:56:10 +00:00

batch_llm_inference.py

[Docs] Improve docstring for ray data llm example (#20597)

2025-07-07 20:06:26 -07:00

chat_with_tools.py

[Doc]: fix typos in Python comments (#24417)

2025-09-08 00:22:16 -07:00

context_extension.py

fix offline inference chat response prompt (#32088)

2026-01-11 14:01:18 +00:00

data_parallel.py

[CI] Fix "2 Node Tests (4 GPUs in total)" (#31090)

2025-12-22 10:32:40 +08:00

disaggregated_prefill.py

Remove deprecated PyNcclConnector (#24151)

2025-09-03 22:49:16 +00:00

encoder_decoder_multimodal.py

[Deprecation] Remove deprecated task, seed and MM settings (#30397)

2025-12-10 19:59:39 -08:00

extract_hidden_states.py

[Spec Decode] Add hidden states extraction system (#33736)

2026-03-02 14:29:09 -05:00

llm_engine_example.py

[Chore]:Extract math and argparse utilities to separate modules (#27188)

2025-10-26 04:03:32 -07:00

llm_engine_reset_kv.py

[Core] Support reseting all running requests' KV while calling reset_prefix_cache (#28827)

2025-12-02 02:25:05 +00:00

load_sharded_state.py

[Quantization][Deprecation] Remove DeepSpeedFp8 (#32679)

2026-01-21 09:32:12 -05:00

lora_with_quantization_inference.py

[Hardware] Replace torch.cuda.empty_cache with torch.accelerator.empty_cache (#30681)

2026-03-04 09:49:47 +00:00

metrics.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

mistral-small.py

[Bugfix] Fix Mistral-small --format (#36782)

2026-03-11 04:47:52 -07:00

mlpspeculator.py

[[V0 deprecation]]Remove VLLM_USE_V1 env (#28204)

2025-11-11 18:22:16 -07:00

multilora_inference.py

fix LoRA-related examples (#29956)

2025-12-04 11:48:30 +08:00

pause_resume.py

[Feat][RL] Pause and Resume with keep requests for single engine (#32351)

2026-02-07 00:08:58 +00:00

prefix_caching.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

prompt_embed_inference.py

Explicitly set return_dict for apply_chat_template (#33372)

2026-01-30 07:27:04 +00:00

qwen_1m.py

Remove V0 attention backends (#25351)

2025-09-21 16:03:28 -07:00

reproducibility.py

[Doc] Update more docs with respect to V1 (#29188)

2025-11-23 10:58:48 +08:00

rlhf_colocate.py

[Hardware] Replace torch.cuda.synchronize() api with torch.accelerator.synchronize (#36085)

2026-03-05 10:36:39 +00:00

rlhf_online_quant.py

Move online quantization to model.load_weights (#26327)

2025-11-18 16:52:41 -08:00

rlhf_utils.py

[Hardware] Replace torch.cuda.synchronize() api with torch.accelerator.synchronize (#36085)

2026-03-05 10:36:39 +00:00

rlhf.py

Move online quantization to model.load_weights (#26327)

2025-11-18 16:52:41 -08:00

routed_experts_e2e.py

Fix routed experts capture for hybrid models (Mamba + Attention) (#35744)

2026-03-11 08:53:10 -07:00

run_one_batch.py

[Misc] Add run one batch script that supports profiling (#32968)

2026-02-10 18:29:49 -08:00

save_sharded_state.py

[Quantization][Deprecation] Remove DeepSpeedFp8 (#32679)

2026-01-21 09:32:12 -05:00

simple_profiling.py

[Cleanup] Refactor profiling env vars into a CLI config (#29912)

2025-12-09 13:29:33 -05:00

skip_loading_weights_in_engine_init.py

[Doc] Add inplace weights loading example (#19640)

2025-07-17 21:12:23 -07:00

spec_decode.py

[Bugfix] Fix --trust-remote-code conflict (#34218)

2026-02-10 00:29:10 -08:00

structured_outputs.py

[Chore] Cleanup guided namespace, move to structured outputs config (#22772)

2025-09-18 09:20:27 +00:00

torchrun_dp_example.py

[CI/Build] Test torchrun with 8 cards (#27548)

2025-10-29 10:26:06 -07:00

torchrun_example.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

vision_language_multi_image.py

[Model] Use mm_position to compute mrope positions for GLM-4.xV (#33039)

2026-02-02 16:55:48 +00:00

vision_language.py

[Models] Intern-S1-Pro (#33636)

2026-02-03 05:49:45 -08:00