Port the MiniMax M3 (text backbone) into vLLM's custom model layout:
- Add MiniMaxM3SparseForCausalLM under vllm/models/minimax_m3/nvidia with the
decoder/model/causal-LM wiring; attention and MoE bodies plus weight loading
are left as stubs. Dense MiniMaxM3MLP is fully ported.
- Add MiniMaxM3SparseForConditionalGeneration as a minimal LM-routing wrapper
(KimiK25-style init_vllm_registered_model on text_config) and register both
architectures.
- Add MiniMaxM3Config (model_type minimax_m3_vl) wrapping MiniMaxM3TextConfig
so config.get_text_config() extracts the backbone; register in the config
registries.
Generalize silu_and_mul_with_clamp to SwiGLU-OAI:
- Add alpha (scales the activation's sigmoid) and beta (added to the
non-activated half) to the CUDA kernel, ops.h, and torch_bindings schema.
Defaults alpha=1.0, beta=0.0 are bitwise-identical to the previous
silu(gate)*up, so existing callers (DeepSeek V4) are unaffected.
- SiluAndMulWithClamp(alpha, beta) used by MiniMaxM3MLP with alpha=swiglu_alpha,
beta=1.0, matching the reference gate*sigmoid(alpha*gate)*(up+1).
AI assistance (Claude) was used for this change.
Signed-off-by: Yongye Zhu <yongye@inferact.ai>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
