[NVIDIA] Fix DeepSeek V4 mHC warmup coverage #1

2026-07-30T08:43:28Z

karylab_agents commented

2026-07-30 08:43:28 +00:00

Summary

expose the NVIDIA DeepSeek V4 decoder hc_pre and hc_post entry points to the warmup detector
warm the first-layer broadcast mHC path before serving
compile the reachable n_splits token-size keys instead of allowing TileLang JIT compilation during inference
add focused unit coverage for detector guards, split-key selection, and broadcast/non-broadcast warmup paths

Why

On the dual-GB10 TP=2 deployment, mHC TileLang kernels were still compiling during live requests. Repeated mhc_pre_*, mhc_post_tilelang, and hc_head_fuse_tilelang compilation could exhaust shared-memory broadcast resources and wedge workers until the RPC timeout. The existing warmup detector silently skipped NVIDIA decoder layers because those layers did not expose hc_pre / hc_post, and the broadcast path was not covered.

Provenance

This is a clean backport of the still-open upstream vLLM PR:

https://github.com/vllm-project/vllm/pull/49707
source commits 32389a6b and 21ef130e

The staged patch ID was verified to be identical to the upstream two-commit patch. It is intentionally separate from the larger SM12x/GB10 preview branch so this root-cause fix can be reviewed independently.

Validation

git diff --check origin/main...HEAD: passed
patch-ID comparison against upstream PR #49707: identical
branch is one commit ahead of the current origin/main with no divergence
local CUDA unit tests were not run: this workstation has no Linux/CUDA vLLM test environment or uv
target validation still required: rebuild the GB10 image, verify startup warmup, and run a long multi-turn TP=2 workload while confirming no mHC TileLang compilation appears after the API becomes ready

AI assistance

Codex assisted with log diagnosis, upstream duplicate-work search, backport preparation, and static verification. A human maintainer should review and own the final merge and GB10 hardware validation.

## Summary - expose the NVIDIA DeepSeek V4 decoder `hc_pre` and `hc_post` entry points to the warmup detector - warm the first-layer broadcast mHC path before serving - compile the reachable `n_splits` token-size keys instead of allowing TileLang JIT compilation during inference - add focused unit coverage for detector guards, split-key selection, and broadcast/non-broadcast warmup paths ## Why On the dual-GB10 TP=2 deployment, mHC TileLang kernels were still compiling during live requests. Repeated `mhc_pre_*`, `mhc_post_tilelang`, and `hc_head_fuse_tilelang` compilation could exhaust shared-memory broadcast resources and wedge workers until the RPC timeout. The existing warmup detector silently skipped NVIDIA decoder layers because those layers did not expose `hc_pre` / `hc_post`, and the broadcast path was not covered. ## Provenance This is a clean backport of the still-open upstream vLLM PR: - https://github.com/vllm-project/vllm/pull/49707 - source commits `32389a6b` and `21ef130e` The staged patch ID was verified to be identical to the upstream two-commit patch. It is intentionally separate from the larger SM12x/GB10 preview branch so this root-cause fix can be reviewed independently. ## Validation - `git diff --check origin/main...HEAD`: passed - patch-ID comparison against upstream PR #49707: identical - branch is one commit ahead of the current `origin/main` with no divergence - local CUDA unit tests were not run: this workstation has no Linux/CUDA vLLM test environment or `uv` - target validation still required: rebuild the GB10 image, verify startup warmup, and run a long multi-turn TP=2 workload while confirming no mHC TileLang compilation appears after the API becomes ready ## AI assistance Codex assisted with log diagnosis, upstream duplicate-work search, backport preparation, and static verification. A human maintainer should review and own the final merge and GB10 hardware validation.

karylab_agents added 1 commit 2026-07-30 08:43:28 +00:00

Fix DeepSeek V4 NVIDIA mHC warmup coverage d789590bb6

Expose the NVIDIA decoder mHC entry points to the warmup detector, warm the broadcast path, and compile the reachable split-key token sizes before serving.

Backport of the fix from vllm-project/vllm#49707 (commits 32389a6b and 21ef130e).

You are not authorized to merge this pull request.

This pull request can be merged automatically.

View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.

git fetch -u origin fix/deepseek-v4-nvidia-mhc-warmup:fix/deepseek-v4-nvidia-mhc-warmup

git checkout fix/deepseek-v4-nvidia-mhc-warmup

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: karylab_agents/vllm#1