Tencent-Hunyuan/Hy-Nomas

Tencent-Hunyuan/Hy-Nomas

Description: Hy-Nomas: Towards Free-form Interactive World Generation

Language: Python

License: NOASSERTION

Stars: 67

Forks: 8

Open issues: 1

Created: 2026-10-21T03:04:54Z

Pushed: 2026-10-23T21:40:36Z

Default branch: main

Fork: no

Archived: no

README:

中文 ｜ English

Hy-Nomas: Towards Free-Form Interactive World Generation

🖥️ Interactive Demo | 🌐 Official Website | 📄 Technical Report (Coming Soon)

https://github.com/user-attachments/assets/e7a2528c-4ccc-461c-8f09-207650f1f0bd

---

Table of Contents

• [Model Introduction](#model-introduction)
• [Key Features](#key-features)
• [Usage](#usage)
• [Environment Setup](#environment-setup)
• [Model Assets](#model-assets)
• [Inference](#inference)
• [Inference with Low-Mem GPU](#inference-with-low-mem-gpu)
• [Interactive Demo](#interactive-demo)
• [LoRA Fine-Tuning](#lora-fine-tuning)
• [SFT Data Preparation](#sft-data-preparation)
• [Offline Sequence Packing](#offline-sequence-packing)
• [Training and Merging](#training-and-merging)
• [Contributing](CONTRIBUTING.md)
• [Security](SECURITY.md)
• [License](#license)
• [Contact Us](#contact-us)

---

Model Introduction

Hy-Nomas is a free-form interactive video world generation model. It supports free-form multi-turn text prompt control or camera control. This repository provides the training, inference, interactive demo, and data annotation code.

Key Features

• Segment-based generation: Autoregressive segment-wise world generation controlled by either segment level prompt or camera control signal, maintaining a lifelong KV cache for consistency.
• Free-form Prompt Control: Control the world generation with any kind of free form user prompt to generate imaginary object, style, scene, and actions.
• Camera control: Condition generation on camera trajectories alongside text prompts.
• Distilled inference: Inference uses 8 denoising steps and a finetuned PixelUMM decoder for 32-channel DCAE-V latents.
• Interactive demo: A web interface supports generation across multiple turns and decoding at segment boundaries.
• Fine-tuning and data preparation: Lora fine-tuning with customized video dataset, and prepare training data from local videos with our annotation workflow.

Usage

Environment Setup

Choose one of the following two setup options. We strongly recommend the pre-packaged Docker image, which provides the Python environment, PyTorch, and the required CUDA attention kernels together.

Option 1: Pre-packaged Docker Image (Recommended)

Prepare a Linux host with a compatible NVIDIA driver, Docker, and NVIDIA Container Toolkit. Run these commands from the repository root:

export HY_NOMAS_IMAGE="hynomasoss/hynomasoss:cu130"

docker pull "$HY_NOMAS_IMAGE"
docker run --rm -it --gpus all --ipc=host --network=host \
-v "$PWD:/workspace/Hy-Nomas" \
-w /workspace/Hy-Nomas \
"$HY_NOMAS_IMAGE" bash

Run the remaining commands inside the container, using its pre-installed Python environment. Place model assets under the mounted repository or add a volume mount for your model directory; use the paths visible inside the container in the launch commands below.

Option 2: Manual Installation

Prepare Linux, Python 3.12+, uv, a compatible NVIDIA driver, the CUDA 13.0 Toolkit (nvcc), and a C++ compiler. Run the following from the repository root:

uv sync --extra all
source .venv/bin/activate

The all extra includes PyTorch 2.10.0 (CUDA 13.0), FlashAttention 2, FlashAttention 3, and MagiAttention. pyproject.toml configures the PyTorch CUDA index, the official FA3 source, and disables build isolation for the attention kernels so their builds can use the environment's PyTorch. Kernel compilation may take some time.

Model Assets

The model repository contains DiT generation models, LoRA adapters, video autoencoders and decoders, and Nomas-PE for online prompt enhancement.

Download all model weights from tencent/Hy-Nomas on Hugging Face, preserving the repository directory structure:

export Hy_Nomas_Model_Path=/path/to/Hy-Nomas
hf download tencent/Hy-Nomas --local-dir "${Hy_Nomas_Model_Path}"

Set Hy_Nomas_Model_Path to your download directory once. The commands below reuse its existing value, defaulting to /path/to/Hy-Nomas in a new shell. Click a model name below to view its files on Hugging Face.

| Category | Model asset | Purpose | |:---|:---|:---| | DiT | [Base autoregressive DiT (FP32)](https://huggingface.co/tencent/Hy-Nomas/tree/main/DiT/base_ar) | Non-distilled checkpoint for segment forcing LoRA fine-tuning. | | DiT | [Distilled DiT (FP32)](https://huggingface.co/tencent/Hy-Nomas/tree/main/DiT/distilled) | Checkpoint for fast, 4/6/8-step inference and the interactive demo. Also serves as the base for merging fine-tuned LoRA adapters. | | DiT | [Quantized Distilled DiT (INT8)](https://huggingface.co/tencent/Hy-Nomas/tree/main/DiT/distilled_int8) | Quantized checkpoint for low-memory inference, including single-GPU deployment. | | DiT | [Cinematic LoRA](https://huggingface.co/tencent/Hy-Nomas/tree/main/DiT/lora_weights/cinematic) | LoRA adapter using cinematic data, to be merged into the distilled DiT to enhance the visual quality of generated worlds. | | DiT | [Robotics LoRA](https://huggingface.co/tencent/Hy-Nomas/tree/main/DiT/lora_weights/dual_arm_robotics) | LoRA adapter trained on the RealSource-World real-world robotic-arm dataset, to be merged into the distilled DiT to enhance the robotics simulation capability of generated worlds. | | VAE | [DCAE-V](https://huggingface.co/tencent/Hy-Nomas/tree/main/VAE/DCAE-V) | Video autoencoder for encoding training videos into latents and decoding them. | | VAE | [Fine-tuned PixelUMM Decoder](https://huggingface.co/tencent/Hy-Nomas/tree/main/VAE/PixelUMM_FT) | Decoder for turning generated DCAE-V latents into video frames. | | VAE | [Quantized PixelUMM Decoder (FP8)](https://huggingface.co/tencent/Hy-Nomas/tree/main/VAE/PixelUMM_FT_FP8) | Decoder with FP8 weights for lower memory use and accelerated decoding. | | PE | [Nomas-PE](https://huggingface.co/tencent/Hy-Nomas/tree/main/PE) | Online prompt-enhancement model used by the interactive demo to convert user instructions into structured generation prompts. |

Inference

Run from the repository root with configs/experiment/inference.yaml,...