forked from Karylab-cklius/vllm
Yongye ZhuandClaude Opus 4.8
a3028cebbf
Scaffold the lightning-indexer sparse-attention path: - MiniMaxM3SparseBackend (registered as MINIMAX_M3_SPARSE): block-sparse GQA backend; get_kv_cache_shape serves both the main K/V cache and the single-vector index-key side cache. - MiniMaxM3IndexerCache: side KV cache for per-token index keys, key-only so it uses a single-vector MLAAttentionSpec rather than a K+V FullAttentionSpec. - MiniMaxM3SparseMetadata (+ prefill/decode sub-metadata) and its builder, splitting the batch via split_decodes_and_prefills. - MiniMaxM3SparseImpl: subclasses AttentionImplBase so it can take a custom forward(query, index_query, kv_cache, index_kv_cache); no alibi / sliding window / logits soft cap. forward is a stub pending the kernel port. MiniMaxM3SparseAttention is merged into a single AttentionLayerBase: it owns the projections, per-head QK norm and RoPE, binds the backend + impl, registers the main K/V cache, and holds the index cache. Its forward computes q/k/v and the index q/k, pre-inserts K/V and index-K into their caches, then calls the sparse impl with only the queries. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Yongye Zhu <yongye@inferact.ai> Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>