At concurrency >1000 req/rank the per-step fill loop in
DecodeBenchConnector.start_load_kv became a host-side bottleneck: for
each (request, group, layer) triple it built a block-ID tensor (H2D
sync), allocated a fresh fill tensor, and performed an indexed write.
The fill content is semantically meaningless for the benchmark, so
fill the entire KV cache once at register_kv_caches time using
in-place tensor.fill_ / tensor.normal_. start_fill_kv becomes a no-op.
Scheduler-side bookkeeping is unchanged.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Signed-off-by: Zijing Liu <liuzijing2014@gmail.com>
