forked from Karylab-cklius/vllm
Shengqi ChenandCodex
a32d1bff95
Explain that pre-built CUDA wheels use the architecture lists selected by the release and build pipelines, which may be narrower than the full set vLLM can build from source. Call out CUDA 12.9 architecture-specific wheel coverage, CUDA 13 family-specific targets, and the no-kernel-image error users may see when a wheel does not cover their GPU. Co-authored-by: Codex <codex@openai.com> Signed-off-by: Shengqi Chen <harry-chen@outlook.com>
Using vLLM
First, vLLM must be installed for your chosen device in either a Python or Docker environment.
Then, vLLM supports the following usage patterns:
- Inference and Serving: Run a single instance of a model.
- Deployment: Scale up model instances for production.
- Training: Train or fine-tune a model.