tencent/HY-World-2.0

HY-World 2.0: A Multi-Modal World Model for Reconstructing, Generating, and Simulating 3D Worlds

[English](README.md) | [简体中文](README_zh.md)

"What Is Now Proved Was Once Only Imagined"

🎥 Video

https://github.com/user-attachments/assets/b56f4750-25c9-48fb-83ff-d58526711463

🔥 News

• [May 18, 2026]: 🤗 Open-source World Generation inference code and WorldStereo 2.0 model weights!
• [May 11, 2026]: 🤗 Open-source HY-Pano 2.0 inference code and model weights!
• [April 16, 2026]: 🚀 Release HY-World 2.0 technical report & partial codes!
• [April 16, 2026]: 🤗 Open-source WorldMirror 2.0 inference code and model weights!

📋 Table of Contents

• [📖 Introduction](#-introduction)
• [✨ Highlights](#-highlights)
• [🧩 Architecture](#-architecture)
• [📝 Open-Source Plan](#-open-source-plan)
• [🎁 Model Zoo](#-model-zoo)
• [🤗 Get Started](#-get-started)
• [🔮 Performance](#-performance)
• [🎬 More Examples](#-more-examples)
• [📚 Citation](#-citation)

📖 Introduction

HY-World 2.0 is a multi-modal world model framework for world generation and world reconstruction. It accepts diverse input modalities — text, single-view images, multi-view images, and videos — and produces 3D world representations (meshes / Gaussian Splattings). It offers two core capabilities:

• World Reconstruction (multi-view images / video → 3D): Powered by WorldMirror 2.0, a unified feed-forward model that simultaneously predicts depth, surface normals, camera parameters, 3D point clouds, and 3DGS attributes in a single forward pass.

HY-World 2.0 is an open-source state-of-the-art world model. We released all model weights, code, and technical details to facilitate reproducibility and advance research in this field.

Why 3D World Models?

Existing world models, such as Genie 3, Cosmos, and HY-World 1.5 (WorldPlay+WorldCompass), generate pixel-level videos — essentially "watching a movie" that vanishes once playback ends. HY-World 2.0 takes a fundamentally different approach: it directly produces editable, persistent 3D assets (meshes / 3DGS) that can be imported into game engines like Blender/Unity/Unreal Engine/Isaac Sim — more like "building a playable game" than recording a clip. This paradigm shift natively resolves many long-standing pain points of video world models:

| | Video World Models | 3D World Model (HY-World 2.0) | |--|---|---| | Output | Pixel videos (non-editable) | Real 3D assets — meshes / 3DGS (fully editable) | | Playable Duration | Limited (typically 1 min) | Unlimited — assets persist permanently | | 3D Consistency | No (flickering, artifacts across views) | Native — inherently consistent in 3D | | Real-Time Rendering | Requires per-frame inference; high latency | Consumer GPUs can render in real time | | Controllability | Weak (imprecise character control, no real physics) | Precise — zero-error control, real physics collision, accurate lighting | | Inference Cost | Accumulates with every interaction | One-time generation; rendering cost ≈ 0 | | Engine Compatibility | ✗ Video files only | ✓ Directly importable into Blender / UE / Isaac Engine | | | $\color{IndianRed}{\textsf{Watch a video, then it's gone}}$ | $\color{RoyalBlue}{\textbf{Build a world, keep it forever}}$ |

All above are real 3D assets (not generated videos) and entirely created by HY-World 2.0 -- captured from live real-time interaction.

✨ Highlights

• Real 3D Worlds, Not Just Videos

Unlike video-only world models (e.g., Genie 3, HY World 1.5), HY-World 2.0 generates real 3D assets — 3DGS, meshes, and point clouds — that are freely explorable, editable, and directly importable into Unity / Unreal Engine / Isaac. From a single text prompt or image, create navigable 3D worlds with diverse styles: realistic, cartoon, game, and more.

• Instant 3D Reconstruction from Photos & Videos

Powered by WorldMirror 2.0, a unified feed-forward model that predicts dense point clouds, depth maps, surface normals, camera parameters, and 3DGS from multi-view images or casual videos in a single forward pass. Supports flexible-resolution inference (50K–500K pixels) with SOTA accuracy. Capture a video, get a digital twin.

• Interactive Character Exploration

🧩 Architecture

• Refer to our tech report for more details

A systematic pipeline of HY-World 2.0 — *Panorama Generation* (HY-Pano-2.0) → *Trajectory Planning* (WorldNav) → *World Expansion* (WorldStereo 2.0) → *World Composition* (WorldMirror 2.0 + Splattings Learning) — that automatically transforms text or a single image into a high-fidelity, navigable 3D world (3DGS/mesh outputs).

📝 Open-Source Plan

• [x] Technical Report
• [x] WorldMirror 2.0 Code & Model Checkpoints
• [x] Full Inference Code for World Generation (WorldNav + WorldStereo + World Composition)
• [x] Panorama Generation (HY-Pano 2.0) Model & Code
• [x] World Expansion (WorldStereo 2.0) Model & Code

🎁 Model Zoo

World Reconstruction — WorldMirror Series

| Model | Description | Params | Date | Hugging Face | |-------|-------------|--------|------|--------------| | WorldMirror-2 [new] | Multi-view / video → 3D reconstruction | ~1.2B | 2026 | Download | | WorldMirror-1 | Multi-view / video → 3D reconstruction (legacy) | ~1.2B | 2025 | Download |

Panorama Generation — HY-Pano Series

| Model | Description | Params | Date | Hugging Face | |-------|-------------|--------|------|--------------| | HY-Pano-2 [new] | Text / image → 360° panorama | ~80B | 2026 | Download | | HY-Pano-2-Qwen [new] | Text / image → 360° panorama | ~425M | 2026 | Download |

World Expansion — WorldStereo Series

| Model | Description | Params | Date | Hugging Face | |-----------------|-------------|-----|------|--------------| | WorldStereo-2 [new] | Panorama → 3DGS world | ~17B | 2026 | Download |

We recommend referring to our previous works, WorldStereo and WorldMirror, for background knowledge…