A Survey: Learning Embodied Intelligence from Physical Simulators and World Models

2025-07-01

AI Summary

This paper addresses the weak generalization and poor adaptability of embodied intelligence in sim-to-real transfer. We propose a unified framework that synergistically integrates high-fidelity physics simulators—with their accurate modeling of external environmental dynamics—and world models—which learn compact, predictive internal representations. Our method jointly incorporates predictive coding, representation learning, and reinforcement learning to enable predictive planning and adaptive decision-making within the perception–reasoning–action loop. Key contributions include: (1) a systematic characterization of the complementary synergy between physics simulators and world models, establishing a novel paradigm bridging simulation-based training and real-world deployment; (2) an open-source, continuously updated literature repository comprehensively surveying technical approaches, state-of-the-art advances, and core challenges; and (3) a scalable theoretical and practical framework to enhance autonomy, cross-task generalization, and environmental adaptability of embodied AI systems.

Technology Category

• Intelligent Robots: Embodied AI
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior

Application Category

• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

The pursuit of artificial general intelligence (AGI) has placed embodied intelligence at the forefront of robotics research. Embodied intelligence focuses on agents capable of perceiving, reasoning, and acting within the physical world. Achieving robust embodied intelligence requires not only advanced perception and control, but also the ability to ground abstract cognition in real-world interactions. Two foundational technologies, physical simulators and world models, have emerged as critical enablers in this quest. Physical simulators provide controlled, high-fidelity environments for training and evaluating robotic agents, allowing safe and efficient development of complex behaviors. In contrast, world models empower robots with internal representations of their surroundings, enabling predictive planning and adaptive decision-making beyond direct sensory input. This survey systematically reviews recent advances in learning embodied AI through the integration of physical simulators and world models. We analyze their complementary roles in enhancing autonomy, adaptability, and generalization in intelligent robots, and discuss the interplay between external simulation and internal modeling in bridging the gap between simulated training and real-world deployment. By synthesizing current progress and identifying open challenges, this survey aims to provide a comprehensive perspective on the path toward more capable and generalizable embodied AI systems. We also maintain an active repository that contains up-to-date literature and open-source projects at https://github.com/NJU3DV-LoongGroup/Embodied-World-Models-Survey.

Problem

• How to develop robust embodied intelligence for AGI
• Role of physical simulators in training robotic agents
• Integration of world models for adaptive decision-making

Innovation

• Using physical simulators for safe robot training
• Employing world models for predictive planning
• Integrating simulators and models for AI generalization