forked from Karylab-cklius/vllm
cbfaaeceeb
Remove the `use_deep_gemm_packed_mxfp8` flag and the special MXFP8
quantization branch in `moe_kernel_quantize_input`. MXFP8 activations are
now always quantized to the plain non-swizzled (M, K/32) uint8 UE8M0 scale
layout, and the pack into DeepGEMM's consumed scale layout (int32, MN-major,
TMA-aligned, 4 UE8M0 per int32) is fused directly into the expert-permute
scatter instead of relying on the GEMM's internal repack.
- utils.py: drop the flag + branch; mxfp8 always uses _mxfp8_e4m3_quantize
(non-swizzled).
- config.py / no_dp_ep.py / oracle/fp8.py: remove the flag plumbing.
- deep_gemm_utils.py: add a PACK_UE8M0 path to _fwd_kernel_ep_scatter_2 that
concatenates 4 UE8M0 bytes per int32 and stores MN-major; deepgemm_moe_permute
allocates the TMA-aligned int32 buffer for the uint8 path. Float32 (FP8/FP4)
scales keep the row-major path unchanged.
mm1 now feeds the grouped GEMM pre-packed int32 scales with recipe_a=(1,32),
matching what the mm2 activation-quant path already does (validate-only
transform).
AI assistance (Claude Code) was used for this change.
Tests run on GB200 (SM100):
- Numerical unit test: fused packed scatter matches a torch reference for
data placement and byte-packing; output layout/stride is identical to
per_token_group_quant_fp8_packed_for_deepgemm.
- gsm8k 5-shot, MiniMax-M3-preview:
TP=4 monolithic : exact_match 0.9249 +/- 0.0073
DP=4 + EP : exact_match 0.9325 +/- 0.0069
Co-authored-by: Claude
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>