[Perf] Skip detokenization in online beam search (#46422)

Signed-off-by: Guy Stone <guys@spotify.com> Signed-off-by: Guy Stone <guystone3@gmail.com> Co-authored-by: Claude <noreply@anthropic.com>
2026-06-23 15:46:09 -07:00
parent 0a3e2dbc09
commit 3cc871aaf1
1 changed files with 1 additions and 0 deletions
@@ -61,6 +61,7 @@ class BeamSearchOnlineMixin(ABC):
            logprobs=logprobs_num,
            max_tokens=1,
            temperature=temperature,
+            detokenize=False,
        )
        all_beams = [
            BeamSearchSequence(