OccAnyScene: Towards Unified Indoor-Outdoor 3D Occupancy Predictio

2026-08-09

AI Summary

This work addresses the challenge that existing 3D semantic occupancy prediction methods struggle to handle heterogeneous indoor and outdoor scenes within a unified framework. To this end, we introduce the cross-scene 3D semantic occupancy prediction task and propose OccAnyScene, a novel framework built upon a pretrained foundation model. OccAnyScene leverages pixel-aligned frustum feature aggregation and a frustum-parameterized Gaussian decoding mechanism to adaptively reconstruct scenes with varying camera configurations and spatial scales. Our approach is the first to enable a single model to consistently process diverse environments while preserving both metric consistency and scene adaptability. It achieves state-of-the-art performance with mIoU scores of 59.92% on Occ-ScanNet (indoor) and 23.06% on SurroundOcc-nuScenes (outdoor), setting new benchmarks for cross-scene 3D semantic occupancy prediction.

Abstract

3D occupancy prediction is fundamental to scene understanding, yet existing 3D semantic occupancy methods are typically specialized to fixed scene types and occupancy protocols. We introduce Cross-Scene 3D Semantic Occupancy Prediction, a new task setting which requires a single model to handle heterogeneous indoor and outdoor scenes with varying cameras, spatial ranges, voxel specifications, and semantic taxonomies. This setting poses a fundamental challenge: achieving metric-consistent yet scene-adaptive image-to-3D lifting across varying camera configurations and scene scales. To address this challenge, we propose OccAnyScene, a pixel-frustum-centered Gaussian framework built upon a pretrained depth foundation model. Specifically, the framework employs Pixel-Aligned Frustum Feature Aggregation to construct a camera-aware frustum query for each feature pixel, and Frustum-Parameterized Gaussian Construction to decode each query into multiple Gaussians whose positions and sizes are constrained by the predicted pixel depth and corresponding frustum geometry. OccAnyScene sets new state-of-the-art results, achieving 59.92% mIoU on the indoor Occ-ScanNet and 23.06% mIoU on the outdoor SurroundOcc-nuScenes.

Problem

• 3D occupancy prediction
• indoor-outdoor scenes
• scene understanding
• heterogeneous scenes
• semantic occupancy

Innovation

• 3D occupancy prediction
• cross-scene generalization
• Gaussian representation
• frustum-based lifting
• foundation model