karylab_agents/vllm
1
0
Fork 0
forked from Karylab-cklius/vllm
Code Pull Requests 2 Actions 1 Packages Activity

[Doc] Clarify supported keys for --speculative-config (#40455)

Browse Source 
Signed-off-by: Wangxiaoxiaoa <Wangxiaoxiaoa@users.noreply.github.com>
Co-authored-by: Wangxiaoxiaoa <Wangxiaoxiaoa@users.noreply.github.com>
This commit is contained in:
xiao
2026-04-22 04:36:17 -07:00
committed by GitHub
co-authored by Wangxiaoxiaoa
parent a250f1bd5f
commit ecbe42e991
4 changed files with 105 additions and 8 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/features/speculative_decoding/parallel_draft_model.md
+3 -3
View File
@@ -36,9 +36,9 @@ vllm serve Qwen/Qwen3-4B \
--port 8000 \
--seed 42 \
-tp 1 \
--max_model_len 2048 \
--gpu_memory_utilization 0.8 \
--speculative_config '{"model": "amd/PARD-Qwen3-0.6B", "num_speculative_tokens": 12, "method": "draft_model", "parallel_drafting": true}'
--max-model-len 2048 \
--gpu-memory-utilization 0.8 \
--speculative-config '{"model": "amd/PARD-Qwen3-0.6B", "num_speculative_tokens": 12, "method": "draft_model", "parallel_drafting": true}'
```
## Pre-trained PARD weights