forked from Karylab-cklius/vllm
Fix FP8 dispatch test to match production behavior
use_fp8_dispatch requires the ElasticBuffer to receive FP8 input. In production, this is ensured by pre-quantizing via moe_kernel_quantize_input when is_block_quantized=True. The test was parametrizing use_fp8_dispatch independently of dtype, allowing bf16 input with use_fp8_dispatch=True which triggers a buffer size assertion in DeepEP v2. Fix: - Derive use_fp8_dispatch from dtype (True only for FP8 weights) - Add block_shape=[128, 128] to quant config for FP8 to enable the block quantization path that pre-quantizes input Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
This commit is contained in:
co-authored by
Claude Opus 4.6
parent
a2a4b00f83
commit
515e36da11
@@ -345,7 +345,6 @@ MNKs = [
|
||||
DTYPES = [torch.bfloat16, torch.float8_e4m3fn]
|
||||
|
||||
|
||||
@pytest.mark.parametrize("use_fp8_dispatch", [False, True])
|
||||
@pytest.mark.parametrize("dtype", DTYPES)
|
||||
@pytest.mark.parametrize("m,n,k", MNKs)
|
||||
@pytest.mark.parametrize("num_experts", [32])
|
||||
@@ -354,7 +353,6 @@ DTYPES = [torch.bfloat16, torch.float8_e4m3fn]
|
||||
@multi_gpu_test(num_gpus=2)
|
||||
@requires_deep_ep_v2
|
||||
def test_deep_ep_v2_moe(
|
||||
use_fp8_dispatch: bool,
|
||||
dtype: torch.dtype,
|
||||
m: int,
|
||||
n: int,
|
||||
@@ -365,6 +363,7 @@ def test_deep_ep_v2_moe(
|
||||
workspace_init,
|
||||
):
|
||||
per_act_token_quant = False
|
||||
use_fp8_dispatch = False
|
||||
|
||||
set_random_seed(7)
|
||||
world_size, dp_size = world_dp_size
|
||||
|
||||
Reference in New Issue
Block a user