karylab_agents/vllm

Fork 0

forked from Karylab-cklius/vllm

Files

T

History

4eafc72928 [Audio] Bundle get_generation_prompt() params into SpeechToTextParams (#36268 )

Signed-off-by: Ekagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>

2026-04-22 12:24:18 +08:00

basic.md

[docs] fix unintentional misspellings (#33863 )

2026-02-04 20:50:59 -08:00

multimodal.md

[Refactor] Simplify dummy data generation (#35025 )

2026-02-22 20:55:27 -08:00

README.md

[Docs] Update the name of Transformers backend -> Transformers modeling backend (#28725 )

2025-11-14 16:34:14 +00:00

registration.md

[Docs] Replace all explicit anchors with real links (#27087 )

2025-10-17 02:22:06 -07:00

tests.md

[Docs] Reorganize pooling docs. (#35592 )

2026-03-19 11:25:47 +00:00

transcription.md

[Audio] Bundle get_generation_prompt() params into SpeechToTextParams (#36268 )

2026-04-22 12:24:18 +08:00

README.md

Summary

!!! important Many decoder language models can now be automatically loaded using the Transformers modeling backend without having to implement them in vLLM. See if vllm serve <model> works first!

vLLM models are specialized PyTorch models that take advantage of various features to optimize their performance.

The complexity of integrating a model into vLLM depends heavily on the model's architecture. The process is considerably straightforward if the model shares a similar architecture with an existing model in vLLM. However, this can be more complex for models that include new operators (e.g., a new attention mechanism).

Read through these pages for a step-by-step guide:

!!! tip If you are encountering issues while integrating your model into vLLM, feel free to open a GitHub issue or ask on our developer slack. We will be happy to help you out!