Habitat-GS: A High-Fidelity Navigation Simulator with Dynamic Gaussian Splatting

2026-04-14

AI Summary

This work addresses the limitations of existing embodied AI simulation environments, which often suffer from low visual fidelity and a lack of realistic, dynamic human models, thereby hindering generalization to real-world scenarios. Building upon Habitat-Sim, the authors introduce a high-fidelity navigation simulator that incorporates 3D Gaussian Splatting (3DGS) for the first time in embodied AI, enabling photorealistic real-time rendering and compatibility with diverse 3DGS assets. They further propose a controllable “Gaussian Avatar” module to model dynamic human obstacles. Evaluated on point-goal navigation tasks, the system significantly improves cross-domain generalization, with mixed-domain training yielding optimal performance. Moreover, it demonstrates strong scalability across varying scene complexities and numbers of avatars.

Technology Category

• Intelligent Robots: Embodied AI
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior

Application Category

• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Training embodied AI agents depends critically on the visual fidelity of simulation environments and the ability to model dynamic humans. Current simulators rely on mesh-based rasterization with limited visual realism, and their support for dynamic human avatars, where available, is constrained to mesh representations, hindering agent generalization to human-populated real-world scenarios. We present Habitat-GS, a navigation-centric embodied AI simulator extended from Habitat-Sim that integrates 3D Gaussian Splatting scene rendering and drivable gaussian avatars while maintaining full compatibility with the Habitat ecosystem. Our system implements a 3DGS renderer for real-time photorealistic rendering and supports scalable 3DGS asset import from diverse sources. For dynamic human modeling, we introduce a gaussian avatar module that enables each avatar to simultaneously serve as a photorealistic visual entity and an effective navigation obstacle, allowing agents to learn human-aware behaviors in realistic settings. Experiments on point-goal navigation demonstrate that agents trained on 3DGS scenes achieve stronger cross-domain generalization, with mixed-domain training being the most effective strategy. Evaluations on avatar-aware navigation further confirm that gaussian avatars enable effective human-aware navigation. Finally, performance benchmarks validate the system's scalability across varying scene complexity and avatar counts.

Problem

• embodied AI
• visual fidelity
• dynamic human avatars
• simulation realism
• generalization

Innovation

• 3D Gaussian Splatting
• embodied AI
• dynamic human avatars
• photorealistic simulation
• navigation simulator