3 people 22c6542fa7 [FlashInfer] Support page size >= 128 for trtllm-gen GQA/MQA (#26) ...

Add power-of-2 page sizes >= 128 (128/256/512/1024) to the FlashInfer
backend's supported kernel block sizes. These are served only by the
trtllm-gen dynamic kernel, which requires Blackwell + GQA/MQA
(num_qo_heads // num_kv_heads > 1), not MHA.

Fail fast in FlashInferMetadataBuilder.__init__ when page_size >= 128 is
requested without a usable trtllm-gen path (trtllm disabled, not
Blackwell, or MHA) instead of silently hitting the native wrappers, and
force the prefill path to trtllm so it does not fall back to FA2.

Regenerate docs/design/attention_backends.md for the new block sizes.

AI assistance (Claude) was used for this change.

Co-authored-by: Claude

Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>

2026-06-02 00:30:09 -04:00

..

arch_overview.md

[Docs] Reorganize online serving docs. (#41907)

2026-05-19 14:43:18 +08:00

attention_backends.md

[FlashInfer] Support page size >= 128 for trtllm-gen GQA/MQA (#26)

2026-06-02 00:30:09 -04:00

cuda_graphs_multimodal.md

[MM][CG] Enable encoder Cudagraph for Step3VL (#42224)

2026-05-17 20:19:13 -07:00

cuda_graphs.md

[torch.compile] Refactor Attention Quant Fusion Pass and Remove Boilerplate (#37373)

2026-03-31 14:15:50 -04:00

custom_op.md

[MoE] Move various experts classes to fused_moe/experts/ (#41979)

2026-05-11 07:54:33 +08:00

dbo.md

[MoE Refactor] Create MK for TRTLLM Kernels (#32564)

2026-03-03 10:39:50 -08:00

debug_vllm_compile.md

[vLLM IR] 2/N fused_add_rms_norm and maybe_inplace overload (#36823)

2026-05-01 23:41:15 -04:00

fused_moe_modular_kernel.md

[Feature]: Remove Chunking From FusedMoE (#34086)

2026-03-12 14:24:38 -04:00

fusions.md

[Kernel] Porting fuse_minimax_qk_norm to manual fusion (#43410)

2026-05-26 13:16:03 -07:00

huggingface_integration.md

[Frontend] Rework fastokens integration (#43168)

2026-05-21 15:36:58 -07:00

hybrid_kv_cache_manager.md

[doc] Hybrid KV Cache Manager design doc (#22688)

2025-08-26 20:19:05 +00:00

io_processor_plugins.md

add io_process_plugin for sparse embedding (#34214)

2026-02-28 09:16:37 +00:00

logits_processors.md

[Bugfix][CI] fix typos (#34934)

2026-03-05 17:05:46 +00:00

lora_resolver_plugins.md

[Docs] Update server entrypoint examples (#42077)

2026-05-09 02:03:52 +00:00

metrics.md

[Examples] Resettle Observability examples. (#40123)

2026-04-17 03:13:31 -07:00

mm_processing.md

[1/N] Reorganize multimodal processing code (#32327)

2026-01-14 15:25:31 +00:00

model_runner_v2.md

[Hardware] Replace torch.cuda.synchronize() api with torch.accelerator.synchronize (#36085)

2026-03-05 10:36:39 +00:00

moe_kernel_features.md

[MoE Refactor] Move remaining experts classes to experts directory (#42334)

2026-05-12 09:19:46 -04:00

multiprocessing.md

[Docs] Update design/multiprocessing.md (#30677)

2026-03-04 17:58:59 +00:00

nixl_kv_cache_lease.md

[Docs][PD][NIXL] Lease extension mechanism for blocks on P (#43099)

2026-05-20 08:58:25 +02:00

optimization_levels.md

[Docs] Update server entrypoint examples (#42077)

2026-05-09 02:03:52 +00:00

p2p_nccl_connector.md

[Examples] Resettle Disaggregated examples. (#40759)

2026-05-06 01:20:38 -07:00

paged_attention.md

[doc] fix broken links (#32158)

2026-01-12 10:18:38 +00:00

plugin_system.md

[Deprecation][1/2] Remove items deprecated in v0.18 (#36470)

2026-03-09 03:43:23 -07:00

prefix_caching.md

docs: prefix caching seems quite outdated (#28784)

2026-01-19 11:49:52 -08:00

torch_compile_multimodal.md

[torch.compile][BE][Multimodal] Remove requirement to set_model_tag to avoid cache conflict (#37345)

2026-03-19 17:26:12 +00:00

torch_compile.md

[Frontend] Remap -O to -cc commandline flag (#29557)

2025-11-28 21:51:12 +00:00

vllm_ir.md

[vLLM IR] 2/N fused_add_rms_norm and maybe_inplace overload (#36823)

2026-05-01 23:41:15 -04:00