nvidia/Muse-Spark-Open-816B-A42B-DFlash2

Model Overview

Description:

The NVIDIA Muse-Spark-Open-816B-A42B-DFlash2 model is the DFlash 2 draft checkpoint for Muse-Spark-Open-816B-A42B, a multimodal mixture-of-experts causal language model that produces text and supports long-context, tool-enabled workflows. The NVIDIA Muse-Spark-Open-816B-A42B-DFlash2 model incorporates DFlash 2 speculative decoding with Model Optimizer. DFlash 2 adds a two-tap in-block convolution around every sublayer and a pairwise path selector that walks one coherent path through each position’s top-k candidates.

This model is ready for commercial or non-commercial use.

Third-Party Community Consideration:

The target model, Muse-Spark-Open-816B-A42B, is not owned or developed by NVIDIA. It has been developed and built to a third-party's requirements; see link to Non-NVIDIA Muse-Spark-Open-816B-A42B Model Card. This DFlash 2 draft model was trained by NVIDIA for use with that target model.

License/Terms of Use:

GOVERNING TERMS: Use of the model/checkpoints is governed by the NVIDIA Open Model Agreement.

Additional Information: Meta’s Muse Model License. Built with Muse Spark Open.

Deployment Geography:

Global

Use Case:

Muse-Spark-Open-816B-A42B is intended for evaluation and development of long-context multimodal and agentic text-generation workflows, including tool-assisted coding. This DFlash2 release is intended for DFlash 2–assisted serving of Muse-Spark-Open-816B-A42B rather than as a standalone target model checkpoint.

Release Date:

Huggingface: 10/16/2026 via https://huggingface.co/nvidia/Muse-Spark-Open-816B-A42B-DFlash2

References

NVIDIA Model Optimizer: https://github.com/NVIDIA/Model-Optimizer DFlash: Block Diffusion for Flash Speculative Decoding

Model Architecture:

Architecture Type: Transformer

Network Architecture: 5 dense decoder layers with non-causal sliding-window attention (window 2048)

Total Parameters: 4.38B; the token embedding and output head are shared with the target and not included

Active Parameters: 4.38B (dense, no expert routing)

Vocabulary Size: 202,048

Input:

Input Type(s): Text, Image, Other (Video), Audio

Input Format(s): String, Red, Green, Blue (RGB), Other (Video), Other (Waveform)

Input Parameters: One-Dimensional (1D), Two-Dimensional (2D), Three-Dimensional (3D), One-Dimensional (1D)

Other Properties Related to Input: Requests support temperature, top_p, and top_k parameters. Combined input and output context length is up to 1,000,000 tokens.

Output:

Output Type(s): Text

Output Format: String

Output Parameters: One-Dimensional (1D)

Other Properties Related to Output: The combined input and output context length is up to 1,000,000 tokens.

Our AI models are designed and/or optimized to run on NVIDIA GPU-accelerated systems. By leveraging NVIDIA's hardware (e.g. GPU cores) and software frameworks (e.g., CUDA libraries), the model achieves faster training and inference times compared to CPU-only solutions.

Software Integration:

Supported Runtime Engine(s):

• TRT-LLM

• vLLM

Supported Hardware Microarchitecture Compatibility:

• NVIDIA Blackwell

Preferred Operating System(s):

• Linux

The integration of foundation and fine-tuned models into AI systems requires additional testing using use-case-specific data to ensure safe and effective deployment. Following the V-model methodology, iterative testing and validation at both unit and system levels are essential to mitigate risks, meet technical and functional requirements, and ensure compliance with safety and ethical standards before deployment.

Model Version(s):

v1.0: Muse-Spark-Open-816B-A42B-DFlash2 is the DFlash 2 draft model for Muse-Spark-Open-816B-A42B, trained with a pre-release build of NVIDIA Model Optimizer 0.47 and released in BF16 (bfloat16)

Training and Evaluation Datasets:

Training Dataset:

Link: Nemotron-Post-Training-Dataset-v2, only prompts from the datasets were used for data synthesis (the original responses from GPT were not used), which is then used to train the DFlash modules. Data Modality: Text Text Training Data Size: 1 Billion to 10 Trillion Tokens Data Collection Method by dataset: Hybrid: Automated, Synthetic Labeling Method by dataset: Hybrid: Automated, Synthetic Properties: About 3.48 million multilingual text samples featuring prompts spanning math, code, STEM, and conversational topics. Each sample includes a synthetic response generated by the target model; the corpus mixes responses generated at five reasoning-effort levels, each including its reasoning trace.

Testing Dataset:

Link: SPEED-Bench, MT-Bench Data Collection Method by dataset: Hybrid: Automated, Manually-Collected, Synthetic Labeling Method by dataset: Hybrid: Automated, Manually-Labeled, Synthetic Properties: The SPEED-Bench qualitative split (880 text prompts in 11 task categories, mostly English), the 32k-token bucket of the SPEED-Bench throughput split (first 100 prompts of each of its three categories), and MT-Bench (80 English two-turn text questions in 8 categories) were used during development to measure acceptance length and select checkpoints.

Evaluation Dataset:

The model is evaluated on the SPEED-Bench and MT-Bench datasets for speculative decoding benchmarking.

Data Collection Method by dataset: Hybrid: Automated, Manually-Collected, Synthetic Labeling Method by dataset: Hybrid: Automated, Manually-Labeled, Synthetic Properties: All prompts are text and mostly in English. The SPEED-Bench qualitative split has 880 prompts in 11 task categories (such as coding, math, reasoning, and writing, plus a multilingual category covering 23 languages); MT-Bench has 80 two-turn questions in 8 categories. Acceptance length over the...