forked from Karylab-cklius/vllm
[XPU] keep generator state of sycl kernel align with pytorch (#41771)
Signed-off-by: Yan Ma <yan.ma@intel.com> Co-authored-by: Qiming Zhang <qiming1.zhang@intel.com> Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
This commit is contained in:
co-authored by
Qiming Zhang
Kunshang Ji
parent
997132911e
commit
bc03f280c8
@@ -289,6 +289,11 @@ class TopKTopPSampler(nn.Module):
|
||||
torch.ops.vllm.xpu_topk_topp_sampler(
|
||||
random_sampled, logits_to_return, logits, k, p, self.logprobs_mode, seeds
|
||||
)
|
||||
# The custom XPU sampler kernel consumes RNG values internally, so advance
|
||||
# the default generator's offset to keep future draws deterministic.
|
||||
offset += logits.numel()
|
||||
state.view(torch.int64)[1] = offset
|
||||
generator.set_state(state)
|
||||
return random_sampled, logits_to_return
|
||||
|
||||
|
||||
|
||||
Reference in New Issue
Block a user