DSOcc: Leveraging Depth Awareness and Semantic Aid to Boost Camera-Based 3D Semantic Occupancy Prediction

2025-05-27

AI Summary

Existing monocular 3D semantic occupancy prediction methods rely on explicit occupancy classification, leading to feature misalignment and limited category learning due to sparse per-class annotations. To address these issues, we propose an implicit depth-aware modeling framework. First, we introduce a novel non-learnable, geometry- and depth-prior-driven soft occupancy confidence computation—replacing explicit classification with end-to-end differentiable implicit state inference. Second, we fuse multi-frame pretrained semantic segmentation features with voxelized occupancy probabilities to enhance cross-frame semantic consistency and class-wise robustness. Third, we adopt camera-centric voxelization, balancing geometric plausibility and computational efficiency. Evaluated on SemanticKITTI, our method achieves state-of-the-art performance among purely vision-based approaches, significantly improving occupancy completeness, geometric accuracy, and semantic classification precision.

Technology Category

• Computer Vision: 3D Computer Vision
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning
• Intelligent Robots: State Estimation

Application Category

• Semantics and Knowledge: Methods, algorithms and applications for the development of semantic models, knowledge graphs and other forms of structured data models with machine-interpretable semantics
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Camera-based 3D semantic occupancy prediction offers an efficient and cost-effective solution for perceiving surrounding scenes in autonomous driving. However, existing works rely on explicit occupancy state inference, leading to numerous incorrect feature assignments, and insufficient samples restrict the learning of occupancy class inference. To address these challenges, we propose leveraging Depth awareness and Semantic aid to boost camera-based 3D semantic Occupancy prediction (DSOcc). We jointly perform occupancy state and occupancy class inference, where soft occupancy confidence is calculated through non-learning method and multiplied with image features to make the voxel representation aware of depth, enabling adaptive implicit occupancy state inference. Rather than focusing on improving feature learning, we directly utilize well-trained image semantic segmentation and fuse multiple frames with their occupancy probabilities to aid occupancy class inference, thereby enhancing robustness. Experimental results demonstrate that DSOcc achieves state-of-the-art performance on the SemanticKITTI dataset among camera-based methods.

Problem

• Improving 3D semantic occupancy prediction accuracy in autonomous driving
• Addressing incorrect feature assignments in occupancy state inference
• Enhancing robustness in occupancy class inference using semantic segmentation

Innovation

• Joint occupancy state and class inference
• Depth-aware voxel representation via soft occupancy
• Semantic segmentation fusion for robust class inference