MobileOcc: A Human-Aware Semantic Occupancy Dataset for Mobile Robots

2025-11-20

AI Summary

To address the lack of dense 3D semantic occupancy perception for mobile robots in crowded pedestrian environments, this paper introduces the first dataset jointly annotated with static object and dynamic human 3D occupancy. We propose the first deformable human mesh joint optimization framework that fuses monocular/stereo images and LiDAR point clouds. Our method enables end-to-end co-optimization of 2D human geometry reconstruction and 3D point cloud geometry refinement—achieving multi-view-consistent occupancy prediction, 3D pose estimation, and pedestrian velocity forecasting. Building upon this, we establish two novel benchmark tasks—occupancy prediction and velocity prediction—with a unified evaluation protocol and multiple baseline models. Extensive experiments validate annotation robustness and demonstrate strong generalization across diverse datasets. The proposed approach significantly enhances robotic spatial understanding of dynamic human environments.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Computer Vision: Motion & Tracking

Application Category

• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

Dense 3D semantic occupancy perception is critical for mobile robots operating in pedestrian-rich environments, yet it remains underexplored compared to its application in autonomous driving. To address this gap, we present MobileOcc, a semantic occupancy dataset for mobile robots operating in crowded human environments. Our dataset is built using an annotation pipeline that incorporates static object occupancy annotations and a novel mesh optimization framework explicitly designed for human occupancy modeling. It reconstructs deformable human geometry from 2D images and subsequently refines and optimizes it using associated LiDAR point data. Using MobileOcc, we establish benchmarks for two tasks, i) Occupancy prediction and ii) Pedestrian velocity prediction, using different methods including monocular, stereo, and panoptic occupancy, with metrics and baseline implementations for reproducible comparison. Beyond occupancy prediction, we further assess our annotation method on 3D human pose estimation datasets. Results demonstrate that our method exhibits robust performance across different datasets.

Problem

• Addressing semantic occupancy perception gaps for mobile robots
• Developing human-aware occupancy modeling in crowded environments
• Establishing benchmarks for occupancy and pedestrian velocity prediction

Innovation

• Mesh optimization framework for human occupancy modeling
• Reconstructs deformable human geometry from 2D images
• Refines human geometry using LiDAR point data