nvidia/Cosmos-AnomalyGen-Metal-Nano

Model Overview

Description:

Cosmos AnomalyGen — Metal Surface (UC2) generates synthetic magnetic-tile-surface anomaly images by inpainting a user-supplied binary mask onto a clean reference tile image, conditioned on one of five trained defect types (Blowhole, Break, Crack, Fray, Uneven). The release ships only the few-shot-finetuned modules — a set of per-defect-type LoRA adapters — which plug into the frozen Cosmos3-Nano diffusion backbone (also using a frozen Wan2.2 VAE tokenizer and a frozen Qwen3-VL text encoder) at inference time. Cosmos AnomalyGen — UC2 v1.2.0 was developed by NVIDIA as part of the Cosmos AnomalyGen pipeline. This model is ready for commercial or non-commercial use.

License/Terms of Use:

GOVERNING DOWNLOAD TERMS: Use of the model is governed by the OpenMDW License Agreement, version 1.1.

ADDITIONAL INFORMATION: The Cosmos3-Nano base model is governed by OpenMDW License Agreement, version 1.1. The training framework uses components governed by Apache License, Version 2.0.

Deployment Geography:

Global

Use Case:

Industrial visual-inspection teams responsible for magnetic-tile / metal-surface QA who have very few (≤5 per defect type) real anomaly examples. The model produces large-scale synthetic anomaly datasets (clean tile + binary mask → realistic Blowhole / Break / Crack / Fray / Uneven image) for training downstream defect-detection or segmentation models, including downstream consumers via the pseudo-labeling export path (COCO instance annotations and per-class classification layout).

Release Date:

Github 10/20/2026 via https://github.com/NVIDIA/cdf-anomalygen

References(s):

• Cosmos Framework — https://github.com/NVIDIA/cosmos-framework
• Cosmos3-Nano — https://huggingface.co/nvidia/Cosmos3-Nano
• Wan2.2 VAE tokenizer — https://huggingface.co/Wan-AI/Wan2.2-TI2V-5B
• Qwen3-VL-8B-Instruct tokenizer — https://huggingface.co/Qwen/Qwen3-VL-8B-Instruct
• LoRA: Low-Rank Adaptation of Large Language Models — paper: https://arxiv.org/abs/2106.09685
• Magnetic Tile Defect dataset — https://github.com/abin24/Magnetic-tile-defect-datasets

Model Architecture:

Architecture Type: Transformer (diffusion MoT backbone with learnable per-defect-type conditioning modules)

Network Architecture:

• per_class_lora *(trainable, included in this release)*: per-defect-type LoRA adapters (rank 8, alpha 8) on the four generation-expert attention projections (q_proj_moe_gen, k_proj_moe_gen, v_proj_moe_gen, o_proj_moe_gen) of all 36 layers — five pairs trained for UC2: metal_surface+MT_Blowhole, metal_surface+MT_Break, metal_surface+MT_Crack, metal_surface+MT_Fray, metal_surface+MT_Uneven; each generation token is routed to the adapter of its defect type.
• tokenizer *(frozen, not redistributed in this release)*: Wan2.2 VAE (16× spatial compression, 48 latent channels); weights are loaded from the separately downloaded Wan-AI/Wan2.2-TI2V-5B checkpoint at inference time.
• text_encoder *(frozen, not redistributed in this release)*: the Qwen3-VL-8B reasoner tower of Cosmos3-Nano, with the Qwen/Qwen3-VL-8B-Instruct tokenizer.
• These modules condition the frozen Cosmos3-Nano Mixture-of-Transformers (MoT) diffusion denoiser (36 layers, hidden size 4096) at inference time.

This model was developed based on Cosmos3-Nano.

Number of model parameters: Approximately 3.8×10^7 (38.3 million) trainable parameters in the released modules — per_class_lora ≈ 7.67M per + pair (rank-8 adapters on 4 projections × 36 layers) × 5 pairs = 38,338,560. The trainable modules are distributed as the model/iter_000014000.pt checkpoint file. The frozen Cosmos3-Nano base contributes ~1.6×10^10 (16 billion) parameters used at inference time but not redistributed in this release.

Input(s):

Input Type(s): Image, Binary Mask, Text

Input Format(s):

• Image: PNG / JPG, Red, Green, Blue (RGB)
• Binary Mask: PNG / JPG, single-channel binary (0 = background, 255 = anomaly region; binarized at threshold 127)
• Text: anomaly-type string in the form + (one of metal_surface+MT_Blowhole, metal_surface+MT_Break, metal_surface+MT_Crack, metal_surface+MT_Fray, metal_surface+MT_Uneven)

Input Parameters:

• Image: Two-Dimensional (2D)
• Mask: Two-Dimensional (2D)
• Text: One-Dimensional (1D)

Other Properties Related to Input: Input clean image and paired mask must have the same dimensions; the model was trained on defect-centred crops at variable model input sizes (256 / 512 / 1024 / 1536 px) and inference is run with the same crop geometry, so there is no fixed input resolution. anomaly_type must exactly match one of the five pairs trained for this UC2 checkpoint — passing an unsupported defect string is rejected by anomalygen/scripts/texture/generate.py against this checkpoint's recipe anomaly_types (the checkpoint carries a geometry stamp and refuses a recipe that differs from the one it trained under). The mask should ideally cover a contiguous defect region that resembles the trained mask distribution; the optional Automatic Mask Placement (AMP) tool, or the default Agentic Mask Generation (AMG) stage, can constrain placement to legal ROIs.

Output(s)

Output Type(s): Image

Output Format(s): PNG; Red, Green, Blue (RGB)

Output Parameters: Two-Dimensional (2D)

Other Properties Related to Output: RGB synthetic anomaly image at the native resolution of the input clean image. Anomaly content is generated inside the user-supplied mask region; the inpainted patch is pasted back onto the clean reference image so non-masked pixels remain identical to the input. Optionally Poisson blending can be enabled. Generation metadata (per-sample guidance, crop_ratio, seed, etc.) is written to texture_ft_generation_result.csv alongside the images; samples rejected by the default content-safety guardrail are listed in guardrail_blocked.csv.

Our AI models are designed and/or optimized to run on NVIDIA GPU-accelerated systems. By leveraging NVIDIA's hardware (e.g. GPU cores) and software frameworks (e.g., CUDA libraries), the model achieves faster training and inference times compared to CPU-only solutions.

Software Integration:

Runtime Engine(s):

• PyTorch (via the Cosmos Framework / Cosmos3-Nano pipeline)
• Cosmos AnomalyGen scripts...