Skip compile SP pass under enforce_eager (V2 runner)

sp_threshold force-enabled compilation_config.pass_config.enable_sp, which the
V2 model runner rejects ("does not yet support: sequence parallelism"). The
eager in-forward SP path reads sp_threshold directly and needs no compile pass;
under enforce_eager there is no compilation to attach it to. Guard the mapping.

Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
This commit is contained in:
Woosuk Kwon
2026-06-30 15:27:09 +00:00
parent 44c1a8eb78
commit e0bbe8d667
+5 -1
View File
@@ -1206,7 +1206,11 @@ class VllmConfig:
# None -> defer to the optimization-level default
# N > 0 -> force the pass on with N as its min-token threshold
sp_threshold = self.parallel_config.sp_threshold
if sp_threshold is not None:
# Only enable the compile-based SP pass when compilation actually runs.
# The eager in-forward SP path reads sp_threshold directly and runs the
# V2 model runner, which rejects the compile SP pass; under enforce_eager
# there is no compilation to attach it to anyway.
if sp_threshold is not None and not self.model_config.enforce_eager:
pass_config.enable_sp = True
if pass_config.sp_min_token_num is None:
pass_config.sp_min_token_num = sp_threshold