Humanoid-OmniOcc: Stereo-Based Full-View Occupancy Dataset for Embodied AI

2026-06-22

AI Summary

Existing voxel-level occupancy prediction datasets are primarily designed for autonomous driving and are limited by front-facing viewpoints, far-field geometry assumptions, and static road priors, rendering them inadequate for the perceptual demands of humanoid robots in complex indoor environments. To address this gap, this work introduces the first panoramic stereo vision occupancy dataset tailored for humanoid robots, encompassing 15 simulated and 5 real-world indoor scenes. Furthermore, it proposes a Real2Sim2Real closed-loop optimization framework coupled with a stereo-depth-prior-guided 2D-to-3D occupancy modeling approach. Experimental results demonstrate that the proposed method significantly outperforms monocular baselines in both simulated and real environments and exhibits strong generalization capabilities in unseen scenes, thereby validating the effectiveness of the dataset design and modeling strategy.

Abstract

Occupancy prediction at voxel-level granularity is essential for safe robotic navigation and interaction in complex environments. Existing occupancy datasets, however, are predominantly designed for autonomous driving with vehicle-centric biases -- forward-facing cameras, far-field geometry, and static road priors -- limiting their applicability to embodied humanoid perception. We present Humanoid-OmniOcc, a large-scale panoramic stereo-based occupancy dataset tailored for humanoid robots. The dataset encompasses 15 diverse simulated indoor scenes and 5 real-world environments, yielding over 155K samples with broad scene and style diversity. Importantly, the dataset is designed around a Real2Sim2Real closed-loop paradigm: real sensor specifications drive physically accurate simulation, simulation produces large-scale annotated training data, and models trained in simulation are directly evaluated on real-world captures -- enabling iterative refinement of the sim-to-real pipeline. We further propose \textbf{H}umanoid \textbf{S}urround \textbf{S}tereo-guided \textbf{Occ}upancy model (Humanoid-OmniOcc) that exploits robust depth priors for accurate 2D-to-3D lifting. Extensive experiments show that Humanoid-OmniOcc consistently outperforms monocular baselines and generalizes well to both unseen simulated test scenes and real-world environments, validating the effectiveness of the Real2Sim2Real design. Code and data will be available upon acceptance at https://d-robotics-ai-lab.github.io/humanoid-omniocc.

Problem

• occupancy prediction
• humanoid robots
• embodied AI
• sim-to-real
• stereo-based dataset

Innovation

• occupancy prediction
• stereo vision
• Real2Sim2Real
• humanoid robotics
• sim-to-real transfer