Real-world grounding in agentic AI

Four approaches to grounding AI agents in the physical world - Amazon Science

Research areas

Our scientific contributions

Publications Research from our scientists and collaborators.

Conferences Our experts present and discuss cutting-edge research at scientific meetings globally.

Research areas

Our scientific contributions

Publications Research from our scientists and collaborators.

Conferences Our experts present and discuss cutting-edge research at scientific meetings globally.

News & blog

The latest from Amazon researchers

Amazon Science Blog Technical deep-dives and perspectives from our scientists.

News Research milestones and recent achievements.

The latest from Amazon researchers

Amazon Science Blog Technical deep-dives and perspectives from our scientists.

News Research milestones and recent achievements.

Amazon Research Awards

Amazon Nova AI Challenge

Research collaborations

Amazon Research Awards

Amazon Nova AI Challenge

Research collaborations

Resources

AGI Labs Meet the team building useful AI agents.

Amazon Nova Try Amazon’s frontier foundation models.

AGI Labs Meet the team building useful AI agents.

Amazon Nova Try Amazon’s frontier foundation models.

Careers Explore our open roles.

Amazon Scholars Faculty research opportunities on industry-scale technical challenges.

Postdoctoral Science Program Early-career research opportunities alongside experienced industry scientists.

Careers Explore our open roles.

Amazon Scholars Faculty research opportunities on industry-scale technical challenges.

Postdoctoral Science Program Early-career research opportunities alongside experienced industry scientists.

Real-world grounding in agentic AI

Four approaches can dramatically improve the performance and trustworthiness of AI agents in operational environments.

June 8, 2026

7 min read

Share

Share

分享到微信

x

Overview by Amazon Nova

• Physics-guided deep learning integrates physical principles into foundation models to ensure predictions obey governing physical laws and require less data for satisfactory accuracy.
• Uncertainty-aware reasoning uses a framework called UQ4CT to produce calibrated uncertainty, allowing AI agents to halt or request human intervention when internal uncertainty exceeds a safety threshold.
• The adapting-while-learning (AWL) framework bridges the text-to-numerical gap by distilling knowledge from physical simulators and dynamically calling specialized tools when tasks exceed the model's training, achieving 29% higher accuracy on physical-science datasets.
• Verifier-augmented grounding uses external software to ensure AI models work within the bounds of logic and reality, refining reasoning through interactive loops and formal verification.

Was this answer helpful?

The year 2026 marks a definitive shift in the AI landscape: we have moved from models that simply know to agents that do. Foundation models (FMs) — large Transformer models pretrained with massive datasets and fine-tuned for diverse downstream tasks — have moved far beyond chatbots, coding, and other digital applications. They are now used as the cognitive engines for AI agents in the physical world, where they plan, use tools, and execute multistep tasks across complex, digitally integrated environments, from warehouses and factories to transportation systems and hospitals.

At Amazon, you can see the transition to this new era of "physical AI" in the debut of Project Eluna, an agentic AI model designed to transform how Amazon fulfillment centers operate. To be useful in a high-stakes physical environment, however, an agent needs to be more than fluent in natural language; it needs to be grounded in physical laws and operational constraints.

In particular, we must overcome the challenge of hallucination, which, in virtual environments, takes the form of fabricated information — made-up citations, factual inaccuracies, and logical fallacies, all output with high levels of certainty. In a physical system, such hallucinations can lead to violations of reality, with detrimental consequences. For example, if an agent suggests a robotic path that ignores the momentum and mass of the items being moved, its output could be potentially dangerous to people or result in damage to products or equipment.

In this article, I propose four approaches to grounding AI agents in the physical world, where "grounding" is defined as the integration of external information, including domain-specific datasets, physical principles, and numerical simulations, to contextualize a model's reasoning.

All four approaches can be used separately or in combination, depending on the specific application. Practical implementation of these approaches will not only accelerate the safe and productive use of AI agents but could allow for their further expansion into new domains.

Four pillars of grounding

Project Eluna is an agentic AI model that lives in the cloud and assists operators who manage operations within fulfillment centers via digital dashboards. It’s designed to act with a degree of autonomy, reasoning through complex operational situations and recommending actions to operation managers. It pulls in historical and real-time data — such as the states of conveyor belts or robots — to anticipate bottlenecks and keep operations running smoothly. The four approaches to grounding AI agents that I describe here grew out of my research at the University of California, San Diego, and with the Amazon Fulfillment Technology (AFT) team, and they help ensure that agents like Eluna are physically consistent and operationally reliable.

Four tethers keep AI agents grounded: physics-guided learning, uncertainty-aware reasoning, numerical precision, and formal verification prevent agents from drifting away from reality.

1. Physics-guided deep learning.

Traditional foundation models can learn to mimic statistical patterns in data but often fail to respect the hard constraints of the physical universe, such as the conservation of mass, energy, or momentum. In physics-guided deep learning (PGDL), we integrate first-principle physical knowledge into the foundation model in pretraining. First principles include symmetries, such as inductive biases like rotations and other transformations, and differential equations that could be used, for instance, in a robot’s motion and control. Not only does this ensure that predictions obey governing physical laws, but grounding a model in…