Rho 1 Collapsing The Multimodal Stack

Rho-1: Collapsing the multimodal stack

← Back to Blog

Oct 5, 2026

Rho-1: Collapsing the multimodal stack

Rho-1: Collapsing the multimodal stack

An omni-reasoning model that understands, simulates, and acts.

An omni-reasoning model that understands, simulates, and acts.

Watch the video 1:00

Today, we are releasing a research preview of Rho-1 , our 19B omni-reasoning model trained from scratch. Within a single neural network, it understands and generates text, images, and video, reasons over them, and takes actions. Most AI systems today are agentic pipelines: a central model plans and delegates, handing jobs off to modality-specific specialists. Each handoff adds latency, and each specialist sees only a narrow request, not the full context. Rho-1 collapses that stack. Text, vision, and robotic actions are unified as tokens within a single context window. Transcript of the recorded session. Turn 1. You: Draw a low aerial view of a tall red-and-white striped lighthouse on a rocky headland above the sea, bright midday. Rho-1, thinking: Okay, so the user wants a scenic coastal image featuring a prominent lighthouse. This is all about crisp maritime clarity and striking architectural contrast. The main creative decision for me is composition: balancing the grand lighthouse with the sweeping coastal landscape below. First, the … Shot plan: This image presents a low-angle aerial view of a striking coastal scene, dominated by a tall, red-and-white striped lighthouse standing on a rugged, rocky headland that juts into the sea. The lighthouse, a … It renders image:0. Turn 2. You: Put a box around the lighthouse. Rho-1, thinking: The user wants the lighthouse marked on image:0. I'll mark the lighthouse on image:0. It draws a box around the lighthouse. Turn 3. You: Animate it: the drone slowly flies toward the lighthouse. Rho-1, thinking: The user wants image:0 animated into a video where the drone approaches the lighthouse. I'll generate video:0 starting from the frame of image:0 and animate the drone moving steadily forward, descending slightly as it approaches the tall … Shot plan: An aerial drone slowly and steadily approaches a tall, red and white striped lighthouse standing prominently on a rugged, rocky outcrop, revealing the surrounding ocean and coastal rocks as it descends … It renders video:0. Turn 4. You: Edit the video: make it a heavy snowstorm, with white snow settling on the rocks and grass in natural colors. Keep the lighthouse, the house and the camera move exactly the same. Rho-1, thinking: The user wants video:0 edited into a heavy snowstorm while keeping everything else intact. I'll generate video:1 from the same scene as video:0: a lighthouse on a rocky cliff above the ocean, with a small white building beside it. The … Shot plan: Edit video:0: keep the same scene, lighthouse, white building, and camera movement, but make it a heavy snowstorm with white snowflakes falling steadily, covering the rocky cliff face, grassy vegetation, and … It renders video:1. Turn 5. You: What changed between the two videos? Rho-1, thinking: The user asks what changed between the videos. Answer in text: the first video was clear, sunny, with falling snow; the second is a snowy storm version. Answer: The first video shows a clear, sunny day with the lighthouse, white house, and blue ocean. The second video is a snowy version of the same scene, with snow falling across the sky and accumulating on the rocks, cliff, and grass, and the overall tone is muted and overcast. The conversation Replay Draw a low aerial view of a tall red-and-white striped lighthouse on a rocky headland above the sea, bright midday. Rho-1 Thinking Okay, so the user wants a scenic coastal image featuring a prominent lighthouse. This is all about crisp maritime clarity and striking architectural contrast. The main creative decision for me is composition: balancing the grand lighthouse with the sweeping coastal landscape below. First, the … Shot plan This image presents a low-angle aerial view of a striking coastal scene, dominated by a tall, red-and-white striped lighthouse standing on a rugged, rocky headland that juts into the sea. The lighthouse, a …

Put a box around the lighthouse. Rho-1 Thinking The user wants the lighthouse marked on image:0. I'll mark the lighthouse on image:0. lighthouse

Animate it: the drone slowly flies toward the lighthouse. Rho-1 Thinking The user wants image:0 animated into a video where the drone approaches the lighthouse. I'll generate video:0 starting from the frame of image:0 and animate the drone moving steadily forward, descending slightly as it approaches the tall … Shot plan An aerial drone slowly and steadily approaches a tall, red and white striped lighthouse standing prominently on a rugged, rocky outcrop, revealing the surrounding ocean and coastal rocks as it descends …

Edit the video: make it a heavy snowstorm, with white snow settling on the rocks and grass in natural colors. Keep the lighthouse, the house and the camera move exactly the same. Rho-1 Thinking The user wants video:0 edited into a heavy snowstorm while keeping everything else intact. I'll generate video:1 from the same scene as video:0: a lighthouse on a rocky cliff above the ocean, with a small white building beside it. The … Shot plan Edit video:0: keep the same scene, lighthouse, white building, and camera movement, but make it a heavy snowstorm with white snowflakes falling steadily, covering the rocky cliff face, grassy vegetation, and …

What changed between the two videos? Rho-1 Thinking The user asks what changed between the videos. Answer in text: the first video was clear, sunny, with falling snow; the second is a snowy storm version. The first video shows a clear, sunny day with the lighthouse, white house, and blue ocean. The second video is a snowy version of the same scene, with snow falling across the sky and accumulating on the rocks, cliff, and grass, and the overall tone is muted and overcast.

Inside the model

State the KV cache

Text reasoning, answers, boxes

Decoder latents to pixels

Text head predicts the next token

Diffusion head predicts the way back to clean

Block 1

Block 1

Block 2

Block 2

Block 3

Block 3

Block 4

Block 4

Block 5

Block 5 ⋮ ⋮

Block N

Block N

shared attention

Understanding stream Generation stream

Text tokens yours and its own

Visual tokens images, video

Noisy latents the image or clip being made

×8

The session Five...