forked from Karylab-cklius/vllm
Cursor AgentandSimon Mo
89f6318db2
- Add comprehensive PRD with current state analysis, gaps, competitive research - Add design sketch with architecture, component design, code sketches - Document hardware topology, model metadata, anonymized workload requirements - Update usage_stats.md with links to V2 design docs Co-authored-by: Simon Mo <simon@inferact.ai>
Using vLLM
First, vLLM must be installed for your chosen device in either a Python or Docker environment.
Then, vLLM supports the following usage patterns:
- Inference and Serving: Run a single instance of a model.
- Deployment: Scale up model instances for production.
- Training: Train or fine-tune a model.