vllm/examples/offline_inference at a5cfbab3c8d53dbf7cb76edfb6ca1264342bf6ed - vllm

basic

2025-02-20 04:53:51 -08:00

openai

[CI/Build] Auto-fix Markdown files (#12941 )

2025-02-08 04:25:15 -08:00

profiling_tpu

Update deprecated Python 3.8 typing (#13971 )

2025-03-02 17:34:51 -08:00

audio_language.py

[Bugfix] Fix LoRA extra vocab size (#15047 )

2025-03-18 09:40:29 -07:00

chat_with_tools.py

[Misc] Add SPDX-License-Identifier headers to python source files (#12628 )

2025-02-02 11:58:18 -08:00

cpu_offload_lmcache.py

[Feature] Support KV cache offloading and disagg prefill with LMCache connector. (#12953 )

2025-02-25 00:38:42 -08:00

data_parallel.py

[BugFix] Fix torch distributed stateless PG backend init (#14870 )

2025-03-15 20:26:19 -07:00

disaggregated_prefill_lmcache.py

Correct capitalisation: VLLM -> vLLM (#14562 )

2025-03-10 16:36:21 +00:00

disaggregated_prefill.py

[Bugfix] Offline example of disaggregated prefill (#13214 )

2025-02-13 20:20:47 -08:00

distributed.py

Update deprecated Python 3.8 typing (#13971 )

2025-03-02 17:34:51 -08:00

eagle.py

[Bugfix] EAGLE output norm bug (#14464 )

2025-03-15 06:50:33 +00:00

encoder_decoder_multimodal.py

[Misc] Add --seed option to offline multi-modal examples (#14934 )

2025-03-17 03:00:17 -07:00

encoder_decoder.py

[Misc] Add SPDX-License-Identifier headers to python source files (#12628 )

2025-02-02 11:58:18 -08:00

llm_engine_example.py

Deprecate best_of Sampling Parameter in anticipation for vLLM V1 (#13997 )

2025-03-05 20:22:43 +00:00

lora_with_quantization_inference.py

[Misc] Clean up the BitsAndBytes arguments (#15140 )

2025-03-20 19:17:12 -07:00

mistral-small.py

[Doc] Update Mistral Small 3.1/Pixtral example (#15184 )

2025-03-20 04:46:06 +00:00

mlpspeculator.py

[V1][Usage] Refactor speculative decoding configuration and tests (#14434 )

2025-03-22 19:28:10 -10:00

multilora_inference.py

Update deprecated Python 3.8 typing (#13971 )

2025-03-02 17:34:51 -08:00

neuron_int8_quantization.py

[Misc] Add SPDX-License-Identifier headers to python source files (#12628 )

2025-02-02 11:58:18 -08:00

neuron.py

[Misc] Add SPDX-License-Identifier headers to python source files (#12628 )

2025-02-02 11:58:18 -08:00

prefix_caching.py

[Misc] Add SPDX-License-Identifier headers to python source files (#12628 )

2025-02-02 11:58:18 -08:00

prithvi_geospatial_mae.py

Update deprecated Python 3.8 typing (#13971 )

2025-03-02 17:34:51 -08:00

profiling.py

Update deprecated Python 3.8 typing (#13971 )

2025-03-02 17:34:51 -08:00

reproduciblity.py

Add an example for reproducibility (#15262 )

2025-03-20 19:55:47 -07:00

rlhf_colocate.py

[RLHF] use worker_extension_cls for compatibility with V0 and V1 (#14185 )

2025-03-07 00:32:46 +08:00

rlhf_utils.py

[RLHF] use worker_extension_cls for compatibility with V0 and V1 (#14185 )

2025-03-07 00:32:46 +08:00

rlhf.py

[RLHF] use worker_extension_cls for compatibility with V0 and V1 (#14185 )

2025-03-07 00:32:46 +08:00

save_sharded_state.py

[Misc] Add SPDX-License-Identifier headers to python source files (#12628 )

2025-02-02 11:58:18 -08:00

simple_profiling.py

[Misc] Add SPDX-License-Identifier headers to python source files (#12628 )

2025-02-02 11:58:18 -08:00

structured_outputs.py

[Misc] Add SPDX-License-Identifier headers to python source files (#12628 )

2025-02-02 11:58:18 -08:00

torchrun_example.py

[Misc] Add SPDX-License-Identifier headers to python source files (#12628 )

2025-02-02 11:58:18 -08:00

tpu.py

[Hardware][TPU] Fix the recompiling issue in logits processor after warmup (#14510 )

2025-03-09 05:44:39 -04:00

vision_language_embedding.py

[Misc] Add --seed option to offline multi-modal examples (#14934 )

2025-03-17 03:00:17 -07:00

vision_language_multi_image.py

[Bugfix] Fix embedding assignment for InternVL-based models (#15086 )

2025-03-20 03:40:13 +00:00

vision_language.py

[Misc] Clean up MiniCPM-V/O code (#15337 )

2025-03-25 10:22:52 +00:00