forked from Karylab-cklius/vllm
Hongxia YangandGitHub
e633514f50
* initial enablement on ROCm Signed-off-by: Hongxia Yang <hongxia.yang@amd.com> * fuse gemma_rmsnorm, swiglu-oai Triton kernels and tune MXFP8 GEMMs for ROCm Replace per-element PyTorch fallbacks with single-pass Triton kernels for the AMD/ROCm MiniMax-M3 path: - gemma_rmsnorm / gemma_fused_add_rmsnorm: fp32 normalize+scale in one kernel, handles non-contiguous qkv.split views (q_norm/k_norm). - swiglu_oai_split: fused gate*sigmoid(alpha*gate)*(up+beta) with optional clamp, used in both dense MLP and native MXFP8 MoE. - MXFP8 GEMM tuning: set num_warps=8 for both the linear and MoE dot_scaled kernels; use hidden_states.dtype instead of hardcoded bf16. Signed-off-by: Hongxia Yang <hongxia.yang@amd.com> * add unit tests for ROCm fused kernels (rmsnorm, swiglu, mxfp8) Signed-off-by: Hongxia Yang <hongxia.yang@amd.com> * [ROCm] Add NATIVE_MXFP8 backend, use supports_mx() for arch gating, fix lint Signed-off-by: Hongxia Yang <hongxia.yang@amd.com> * add adaptive tile to swiglu-oai kernal Signed-off-by: Hongxia Yang <hongxia.yang@amd.com> * [ROCm] mimimax-m3: enable vision-language support, mirroring nvda path Signed-off-by: Hongxia Yang <hongxia.yang@amd.com> * [ROCm] minimax-m3: review feedback on SWIGLUOAI_UNINTERLEAVE and tidy MXFP8 MoE oracle Signed-off-by: Hongxia Yang <hongxia.yang@amd.com> * revert changes to nvda mxfp8 oracle to minimize changes Signed-off-by: Hongxia Yang <hongxia.yang@amd.com> * minor Signed-off-by: Hongxia Yang <hongxia.yang@amd.com> --------- Signed-off-by: Hongxia Yang <hongxia.yang@amd.com>