NVIDIA/cudnn-frontend v1.29.0

v1.29.0 release

Repository: NVIDIA/cudnn-frontend

Tag: v1.29.0

Published: 2026-09-13T05:38:31Z

Prerelease: no

Release notes:

cuDNN Frontend v1.29.0 Release Notes

cuDNN Frontend v1.29.0 is the recommended version for cuDNN 9.26 and later releases.

New: HSTU attention 🚀 🚀

import cudnn
from cudnn.hstu_attention import hstu_attn_varlen_func

HSTU (Hierarchical Sequential Transduction Unit) attention arrives as a complete CuTe DSL kernel family for Blackwell (#487) — packed variable-length forward and backward, FP16/BF16, head dimensions 64, 128, and 256, with full, causal, local, and arbitrary masks, paged-KV forward, and strided or preallocated gradient outputs. HSTU replaces softmax with a SiLU score transformation and derives block-sparse metadata automatically. Explicit-stream execution is allocation- and lifetime-safe, cross-device PyTorch streams are rejected, and output overlap is validated against all read-only metadata.

Built out over the release:

• D32 head dimension (#885) and LMSD — a BF16 LayerNorm-Multiply-SiLU-Dropout forward and backward pair for hidden dimensions below 1024 divisible by 8 (#946), with independently strided X/U inputs, optional SiLU, dropout, concatenated U/X outputs, and optional dWeight. Launch and vector configuration are chosen from the hidden dimension and the runtime device SM count, and changing a row stride does not trigger recompilation. Independent backward output strides followed in #962.
• qlen=1 causal and local attention optimized on SM100, SM103, and SM107 (#900), after a layout refactor that cleaned up the qlen=1 kernels (#927).
• Fixes: TMA store synchronization (#784), block-sparse test input ordering before the cross-stream build (#938), and the removal of the cross-tensor storage overlap checks (#1012).

New: DSA sparse attention forward — the loop closes 🚀 🚀

cudnn.DSA.SparseAttentionForward and sparse_attention_forward_wrapper (#569) add the SM100 sparse forward path, so the DeepSeek Sparse Attention forward/backward workflow now completes inside the frontend-only CuTe DSL API instead of requiring an external FlashMLA forward. H64 D512/D576 and the H128 D512 small-top-k prefill specialization are covered, including arbitrary logical top-k lengths, invalid/out-of-bounds and duplicate indices, per-query lengths, attention sinks, and an optional indexer LSE. Supported on SM100-family capabilities 10.0, 10.3, and 10.7; decode, split-KV, regular H128, SM90, and FP8 cache paths are not included.

On the backward side:

• Deterministic SM100 backward with a bounded-wave dKV reduction (#810).
• Two-CTA specialization for BF16 H128/D512 — 1.10–1.16× faster (#817), an H32/D576 specialization (#760), and optimized indexer-backward gradient kernels (#730).
• Gather4 and score kernels optimized, with sparse attention documented (#957).
• Fixes: three SM90 top-k and attention-sink boundary failures (#785), invalid sparse rows on the SM100 backward (#877), backward boundaries alongside relaxed SM100 BSA FP8 shape limits (#926), and deterministic handling of infinite sinks (#940).

New: Flex Attention 🚀 🚀

import cudnn
mask_plan = cudnn.create_mask_plan(...)
out = cudnn.flex_attn_func(q, k, v, mask_plan)

The experimental cudnn.flex_attention namespace (#775) brings CuTe DSL forward and backward kernels for SM90, SM100, and SM103, covering fixed-length and variable-length MHA/GQA workloads, with compact arbitrary-mask planning, scheduling, runtime compilation and cache support, autograd integration, and lazy top-level exports. The port also syncs the CUTLASS DSL 4.6.0/4.6.1 bulk-copy election fix that otherwise deadlocks the SM90 backward. An SM100 2-CTA mask-slot synchronization bug was fixed in #993.

New: torch.sdpa runs on the cuDNN Python API 🚀 🚀

torch.sdpa is now served end to end — forward and backward, dense and varlen — by the cuDNN Python API (#554).

The `"CUDNN"` provider (python/cudnn/torch/) registers with torch.nn.attention's flash-impl registry (PyTorch 2.13+, the same mechanism FA3/FA4 use) and overrides the CUDA kernels of aten::_scaled_dot_product_cudnn_attention{,_backward}, so F.scaled_dot_product_attention under sdpa_kernel([CUDNN_ATTENTION]) and torch.nn.attention.varlen.varlen_attn run on pygraph plus the engine Router. Registration is passive; activation stays explicit.

Alongside it, cudnn::sdpa_bwd gained dense BHSD backward — it previously served only packed THD and raised NotImplementedError on dense. With that in place the provider's dense backward stops falling back, closing the last C++ hop in a dense training step. Three defects were fixed in the same change: 2·B blocking device-to-host syncs per varlen backward (the host loop is replaced by the device-side thd_lse_to_padded() conversion, which makes the path capturable), dense operands never being normalized to the innermost-dense / 16B-aligned-base contract, and the dense autograd path dropping is_deterministic so use_deterministic_algorithms(True) still built a non-deterministic backward.

The experimental torch op was replaced by cudnn::sdpa_fwd / cudnn::sdpa_bwd (#780), which then gained `window_right` and asymmetric band service (#901).

New: Paged KV caches on the FROST SDPA engine 🚀 🚀

Paged KV caches are served by the existing SDPA graph API and the FROST SM100 engine — no new entry point (#964). A graph built with cuDNN's own paged-cache contract (paged_attention_k_table / paged_attention_v_table plus paged_attention_max_seq_len_kv) now lowers onto the d128 f16/bf16 kernel's new PAGED_KV specialization. FlashInfer's cudnn_batch_decode_with_kv_cache already builds exactly this graph, so with CUDNN_FRONTEND_ENABLE_FROST_ENGINES=1 vLLM and SGLang decode reach the FROST kernel with zero integration work.

• HND ([num_pages, H_kv, page_size, D]) and NHD ([num_pages, page_size, H_kv, D]) page layouts — the layout is only the pool's strides, read off the bound strides, with no new TemplateParams field.
• d128 and d256 f16/bf16 flavors; d=64 rides the d128 envelope and d=192/192 the d256 one.
• THD (ragged) queries over the paged cache — chunked prefill on the same engine.
• Per-batch KV lengths read in-kernel; num_pages / max_pages never enter the compile key. graph.execute...