forked from Karylab-cklius/vllm
Yongye ZhuandClaude Opus 4.8
99bd07204b
Port the MiniMax M3 (text backbone) into vLLM's custom model layout: - Add MiniMaxM3SparseForCausalLM under vllm/models/minimax_m3/nvidia with the decoder/model/causal-LM wiring; attention and MoE bodies plus weight loading are left as stubs. Dense MiniMaxM3MLP is fully ported. - Add MiniMaxM3SparseForConditionalGeneration as a minimal LM-routing wrapper (KimiK25-style init_vllm_registered_model on text_config) and register both architectures. - Add MiniMaxM3Config (model_type minimax_m3_vl) wrapping MiniMaxM3TextConfig so config.get_text_config() extracts the backbone; register in the config registries. Generalize silu_and_mul_with_clamp to SwiGLU-OAI: - Add alpha (scales the activation's sigmoid) and beta (added to the non-activated half) to the CUDA kernel, ops.h, and torch_bindings schema. Defaults alpha=1.0, beta=0.0 are bitwise-identical to the previous silu(gate)*up, so existing callers (DeepSeek V4) are unaffected. - SiluAndMulWithClamp(alpha, beta) used by MiniMaxM3MLP with alpha=swiglu_alpha, beta=1.0, matching the reference gate*sigmoid(alpha*gate)*(up+1). AI assistance (Claude) was used for this change. Signed-off-by: Yongye Zhu <yongye@inferact.ai> Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>