NVIDIA/cudnn-frontend v1.32.0

v1.32.0 release

Repository: NVIDIA/cudnn-frontend

Tag: v1.32.0

Published: 2026-10-27T03:20:31Z

Prerelease: no

Release notes:

cuDNN Frontend v1.32.0 Release Notes

cuDNN Frontend v1.32.0 adds Mamba-2, Qwen sparse attention training, Rubin triangle operations, and a CuTe DSL normalization backend. FROST SDPA gains more native forward and backward configurations, broader paged-cache support, and lower setup overhead. This release also adds FROST plan serialization, expands JAX grouped GEMM support, and updates Python bindings and dependencies.

New: Mamba-2 linear attention 🚀 🚀

Mamba-2 joins the FROST linear-attention family with forward, backward, and state-summary operations on SM100, SM103, and SM107. The Python entry point is cudnn.linear_attention.ops.mamba2 (#1577).

• Supports packed sequences, BF16/FP16 inputs, initial and final recurrent states, optional skip connections, output gating, and step-size controls. The supported state dimension is 128.
• State summaries expose the per-sequence end state and transition needed to compose sequence segments. Summary backward is also available.
• Checkpoints can use checkpoint_dtype=torch.float32; Mamba-2 checkpoint intervals are multiples of 128 tokens (#1694).

The wider linear-attention family also gains persistent caching for common forward kernels, a faster KDA backward pipeline, and fixes for multistream workspace reuse, GDP interleaved layouts and cached summaries, and GDN-2 NaNs under steep per-channel decay (#1708, #1699, #1569, #1664, #1695, #1644).

New: Qwen sparse attention training 🚀 🚀

cudnn.ops.qsa_forward and qsa_backward provide explicit packed QSA training calls for Blackwell SM100/SM103 and Rubin SM107. They support BF16 D256 attention with 12Q/1KV or 24Q/2KV heads and four-token blocks. Forward returns the BF16 output, an FP32 saved output, and natural-log LSE for backward. These functions do not install an autograd wrapper (#1765, #1778).

• Indexed training accepts existing token selections through qsa_indexed_forward and qsa_indexed_backward, initially for BF16 12Q/1KV D256 on SM100/SM103. At 128K tokens with selection width 2052, compact IDs use 256.5 MiB in either direction, compared with 16 GiB for bitmap-forward IDs or 8 GiB for paired bitmap-backward IDs. The indexed path also avoids the 4 GiB selection bitmap. These figures describe selection metadata, not total memory (#1774).
• Packed causal top-k backward is available through selection_mode="causal_topk", with explicit sequence boundaries and a bounded-workspace two-pass implementation. This mode has a stricter selection contract and requires the CUDA 13 CuTe DSL compiler (#1781, #1787).
• QSA attention and selection functions are consolidated under cudnn.ops.qsa_*. Indexed attention consumes caller-supplied selections; the separate selector APIs compute them.

Gated attention block: serving and packed training 🚀 🚀

The experimental Rubin SM107 D256 gated attention block gains paged-cache EXTEND and DECODE modes. These combine projection, optional QK normalization, RoPE, cache writes, attention, gating, and output projection through the block API. Weight loaders cover the documented Hugging Face and vLLM layouts (#1471, #1606, #1619).

• Qwen sparse attention serving: QsaSpec selects the index-list sparse attention path. Supported modes include caller-supplied lists and an integrated compression, scoring, and top-k indexer, including paged decode (#1639, #1789).
• Packed quantized execution: the unfused MXFP8 forward and per-tensor FP8 backward gain packed-sequence support. Packed half forward can fuse the output gate, and supported packed backward plans accept an externally computed delta (#1502, #1505, #1617, #1497).
• Fewer launches: MXFP8 backward fuses small operations and reuses block-scale dQ work. Serving paths add small-row projection kernels, fused preparation/cache writes, and programmatic dependent launch where supported (#1466, #1680, #1763).
• Compiled blocks can update quantization scales between steps through update_quant_scales. The fully fused MXFP8 projection also accepts MXFP4 weights (#1506, #1487).

SDPA: wider native forward support 🚀 🚀

Blackwell SM100/SM103

• Adds native FP16/BF16 Q/K576, V512 attention with dense queries and dense or paged KV, including absorbed-MLA cache layouts. It supports partial GQA packing, split-KV, independent K/V tables and layouts, masks, sinks, and optional Stats. Packed queries remain outside this new flavor (#1902).
• Adds native paged per-tensor FP8 D256 execution and physical-origin support for paged FP8 THD inputs. Paged FP8 attention can use sinks (#1887, #1895, #1874).
• Expands packed GQA prefill and D256 THD split coverage. Paged K and V can use...