nvidia/cdf-sim2real-smt-solderlight

Model Overview

Description:

cdf-sim2real-smt-solderlight transforms Omniverse-rendered synthetic printed-circuit-board (PCB) component crops into restrained raked solder-light automated-optical-inspection (AOI) style images for inspection data augmentation. The release is an NVIDIA fine-tuned version of the Qwen-Image-Edit image-to-image diffusion pipeline (diffusion transformer, Qwen2.5-VL text encoder, Qwen-Image VAE, tokenizer, image processor, and scheduler configuration), specialized for Omniverse-rendered printed-circuit-board (PCB) component-crop → solder-light style transfer. cdf-sim2real-smt-solderlight v1.0.0 was developed by NVIDIA as part of the CDF sim-to-real inspection-data augmentation pipeline. This model is ready for commercial or non-commercial use.

License/Terms of Use:

Governing Terms: Use of this model is governed by the OpenMDW License Agreement, Version 1.1 (OpenMDW-1.1). Additional Information: Apache License, Version 2.0 applies to the upstream Qwen-Image-Edit components.

Deployment Geography:

Global

Use Case:

NVIDIA engineers and researchers building printed-circuit-board (PCB) inspection / automated optical inspection (AOI) systems that need restrained raked solder-light component-crop imagery for data augmentation and sim-to-real workflow development. The model converts Omniverse-rendered synthetic printed-circuit-board (PCB) component crops into the target solder-light AOI style while preserving component identity, crop geometry, markings, and body shape. This model is not intended to be the primary inspection decision-maker; it is a sim-to-real data-translation step. Inspection pass/fail decisions must come from a downstream inspection model with human review.

Release Date:

Github 10/20/2026 via https://github.com/NVIDIA/paidf-augmentation

References(s):

• Qwen-Image-Edit base model — https://huggingface.co/Qwen/Qwen-Image-Edit
• Qwen2.5-VL text encoder — https://huggingface.co/Qwen/Qwen2.5-VL-7B-Instruct
• LoRA: Hu et al., "LoRA: Low-Rank Adaptation of Large Language Models," 2021 — https://arxiv.org/abs/2106.09685
• Diffusers library — https://github.com/huggingface/diffusers

Model Architecture:

Architecture Type: Transformer (diffusion transformer with cross-modal conditioning)

Network Architecture: This release is a self-contained HuggingFace diffusers pipeline directory, located under model/ in this repository. All of the following components are redistributed as part of the release artifact:

• model/transformer/ *(NVIDIA fine-tuned, redistributed)*: the upstream Qwen-Image-Edit flow-matching image-to-image diffusion transformer, fine-tuned by NVIDIA on the attention and feed-forward projections of QwenImageTransformerBlock (to_q, to_k, to_v, to_out.0, the cross-attention add_{q,k,v}_proj / to_add_out, and img_mlp / txt_mlp projections).
• model/text_encoder/ *(redistributed unmodified)*: Qwen2.5-VL; attends over both the input image and the instruction prompt.
• model/vae/ *(redistributed unmodified)*: Qwen-Image VAE.
• model/tokenizer/, model/processor/, model/scheduler/, and model/model_index.json *(redistributed unmodified)*: Qwen-Image-Edit tokenizer, image processor, scheduler configuration, and pipeline entry point.
• Fine-tuning methodology: NVIDIA fine-tuned the transformer using LoRA (rank 16, alpha 16, dropout 0.0, ~1.7 × 10^8 parameters introduced during training). Training used the paired clean-prompt solder-light dataset for 3000 steps with batch size 1, learning rate 1e-4, seed 0, bf16 precision, gradient checkpointing, and Qwen-Image's resolution-aware time-shift schedule. The final LoRA adapter was fused into the base Qwen/Qwen-Image-Edit pipeline at scale 1.0 in bf16, so the released artifact is a standalone diffusers pipeline that requires no separate adapter file at inference time.
• The pipeline can be loaded directly with diffusers.QwenImageEditPipeline.from_pretrained("nvidia/cdf-sim2real-smt-solderlight", subfolder="model"), or by pointing from_pretrained at a local copy of the model/ directory.

This model was developed based on Qwen-Image-Edit.

Number of model parameters: Approximately ~2.0 × 10^10 (20B) total parameters in the released checkpoint. Of these, ~1.7 × 10^8 (170M) LoRA parameters in the diffusion transformer were trained by NVIDIA and then fused into the released transformer; the remaining parameters come from the upstream Qwen-Image-Edit pipeline (transformer + Qwen2.5-VL text encoder + Qwen-Image VAE) and are redistributed unmodified.

Input(s):

Input Type(s): Image, Text

Input Format(s):

• Image: PNG / JPG, Red, Green, Blue (RGB)
• Text: UTF-8 instruction prompt (English)

Input Parameters:

• Image: Two-Dimensional (2D)
• Text: One-Dimensional (1D)

Other Properties Related to Input: Fine-tuned at target area 262,144 pixels (~0.5 MP, approximately 512×512 for square crops); other resolutions are accepted by the underlying diffusers pipeline but NVIDIA fine-tuning was not performed at them, so style fidelity may degrade. Input must be a single Omniverse-rendered synthetic printed-circuit-board (PCB) component crop similar to the solder-light source distribution used during training. The accompanying instruction prompt is the fixed English sentence used during training; the trained prompt is:

> "Render this PCB component crop as a restrained raked solder-light AOI inspection photograph. Keep the exact source geometry, crop, markings, and component body shape. Apply bright orange-red and cobalt-blue specular highlights only to existing solder pads, exposed leads, endcaps, pins, and metallic contacts. Keep plastic or ceramic IC bodies and large matte component centers dark, flat, and non-emissive; no fiery rings, halos, glow, speckled noise, or texture blooms on component bodies. Do not add any new letters, labels, logos, captions, watermarks, borders, or decorative graphics."

The instruction prompt is fixed and not user-configurable. The only input a user provides is the Omniverse-rendered synthetic printed-circuit-board (PCB) image to be processed; the model performs an image-constrained relighting edit of that image, not free-text text-to-image generation. This is a deliberate guardrail: the model was fine-tuned on this single instruction only, so the prompt is locked and cannot be a...