nvidia/NVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-BF16

NVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-BF16

Description:

Nemotron-Labs-3-Puzzle-75B-A9B is a deployment-optimized large language model developed by NVIDIA, derived from Nemotron-3-Super-120B-A12B. The model is produced using Iterative Puzzle, a post-training compression framework, with the goal of significantly improving inference efficiency for interactive, reasoning-heavy, and long-context workloads while preserving strong downstream accuracy.

The model employs a hybrid MoE architecture with interleaved Mamba, MoE, and Attention layers. Like Nemotron-3-Super, it supports Multi-Token Prediction (MTP) for faster text generation. Compared to its parent, Puzzle-75B-A9B reduces the model from 120.7B total / 12.8B active parameters to 75.3B total / 9.3B active parameters.

See the tech report for full training and compression details: Nemotron-Labs-3-Puzzle-75B-A9B: Compressing Hybrid MoE LLMs.

Compared to Nemotron-3-Super, Puzzle-75B-A9B:

• Achieves approximately 2× higher server throughput on a single 8×B200 node at matched user-throughput constraints,
• Increases sustainable 1M-token single-H100 concurrency from 1 request to 8 requests,
• Maintains strong accuracy across reasoning, coding, multilingual, long-context, and agentic benchmarks.

The supported languages include: English, French, German, Italian, Japanese, Spanish, and Chinese.

This model is ready for commercial use.

License/Terms of Use

Governing Download Terms: Use of this model is governed by the OpenMDW License Agreement, version 1.1 (OpenMDW-1.1).

This project is currently not accepting contributions.

Benchmarks

| Benchmark | Nemotron-Labs-3-Puzzle-75B-A9B-BF16 | Nemotron-3-Super-120B-A12B-BF16 | |---|---|---| | General Knowledge | | | | MMLU-Pro | 82.4 | 83.8 | | Reasoning | | | | AIME25 (no tools) | 89.7 | 92.2 | | HMMT Feb25 (no tools) | 93.4 | 94.2 | | HMMT Feb25 (with tools) | 93.9 | 95.5 | | GPQA (no tools) | 78.6 | 80.5 | | GPQA (with tools) | 79.5 | 81.3 | | LiveCodeBench (v5 2024-07↔2024-12) | 81.1 | 82.1 | | SciCode (subtask) | 40.6 | 42.3 | | HLE (no tools) | 16.5 | 18.5 | | Agentic | | | | Terminal Bench (hard subset) | 24.0 | 25.5 | | SWE-Bench (OpenHands) | 56.9 | 59.5 | | TauBench V2 | | | | Airline | 55.8 | 56.9 | | Retail | 63.2 | 64.3 | | Telecom | 61.5 | 61.1 | | Average | 60.2 | 60.8 | | Chat & Instruction Following | | | | IFBench (prompt) | 71.9 | 73.4 | | Scale AI Multi-Challenge | 56.6 | 56.6 | | Arena-Hard-V2 | 68.6 | 72.8 | | Long Context | | | | AA-LCR | 56.9 | 56.8 | | RULER @ 256k | 95.1 | 96.7 | | RULER @ 512k | 94.2 | 95.7 | | RULER @ 1M | 92.2 | 93.9 | | Multilingual | | | | MMLU-ProX (avg over langs) | 77.5 | 79.5 | | WMT24++ (en→xx) | 85.2 | 86.8 |

All evaluation results were collected via Nemo Evaluator SDK and for most benchmarks, the Nemo Skills Harness. For reproducibility purposes, more details on the evaluation settings can be found in the Nemo Evaluator SDK configs folder and the reproducibility tutorial for Nemotron 3 Super. The open source container on Nemo Skills packaged via NVIDIA's Nemo Evaluator SDK used for evaluations can be found here. In addition to Nemo Skills, the evaluations also used dedicated open-source packaged containers for Tau-2 Bench (default prompt), Terminal Bench Hard (48 tasks), ScaleAI Multi Challenge Multi-turn Instruction Following, and Ruler.

The following benchmarks are not onboarded yet in our open source tools and for these we used either their official open source implementation or otherwise an internal scaffolding that we plan to open source in the future: SWE Bench Verified (OpenHands).

Deployment Geography:

Global

Use Case:

NVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-BF16 is a general purpose reasoning and chat model intended to be used in English, Code, and supported multilingual contexts. This model is optimized for collaborative agents and high-volume workloads. It is intended to be used by developers designing AI Agent systems, chatbots, RAG systems, and other AI-powered applications. This model is also suitable for complex instruction-following tasks and long-context reasoning.

Release Date:

July 6, 2026 via Hugging Face

References(s):

• [\[2411.19146\] Puzzle: Distillation-Based NAS for Inference-Optimized LLMs](https://arxiv.org/abs/2411.19146)
• [\[2604.12374\] Nemotron 3 Super: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoning](https://arxiv.org/abs/2604.12374)

Model Architecture:

• Architecture Type: Mamba2-Transformer Hybrid Latent Mixture of Experts (LatentMoE) with Multi-Token Prediction (MTP)
• Network Architecture: Modified Nemotron-3-Super-120B-A12B-BF16 architecture with smaller Mamba SSM state size, varying number of active experts per layer and varying expert intermediate channel size across layers.
• Number of model parameters: 75B Total / 9.3B Active

Model Design

Puzzle-75B-A9B is a compressed variant of Nemotron-3-Super optimized for interactive deployment. We designed the model to maximize server throughput under high user throughput constraints.

The model was constructed using a multi-stage pipeline that combines the Iterative Puzzle compression framework with knowledge distillation, reinforcement learning, quantization, and Multi-Token Prediction head. The compression process jointly optimizes heterogeneous MoE pruning, active parameter budget, and Mamba pruning to improve inference efficiency while preserving model quality. Attention layers are left unchanged because the parent model is already KV-cache efficient.

Compression is applied to three architectural dimensions:

• Heterogeneous MoE Channel Pruning: \

Routed expert intermediate dimensions are pruned...