forked from Karylab-cklius/vllm
mistral0105andClaude
04009ff40b
Populate cache_read_input_tokens and cache_creation_input_tokens in the Anthropic Messages API response, which were previously always None. Key changes: - Add _get_cached_tokens() and _compute_cache_usage() helpers to map vLLM's prefix cache hits to Anthropic's usage format - Fix input_tokens semantics: Anthropic defines total_input = input_tokens + cache_read + cache_creation, so input_tokens must exclude cached tokens (previously it included them) - Set cache_creation_input_tokens to 0 when cache info is available (vLLM's prefix caching only tracks cache reads, not writes) - Force enable_prompt_tokens_details=True for AnthropicServingMessages so cache fields are always populated regardless of CLI flag - Cover all three AnthropicUsage construction sites: non-streaming full response, streaming message_start, and streaming message_delta Fixes #33923 Co-authored-by: Claude Signed-off-by: mistral0105 <zhangshuoming17@mails.ucas.ac.cn>