$alpha$-OCC: Uncertainty-Aware Camera-based 3D Semantic Occupancy Prediction

2024-06-16

AI Summary

To address the lack of model uncertainty modeling in 3D semantic occupancy prediction (OCC)—which undermines the reliability of autonomous driving planning and mapping—this paper proposes Depth-UP, the first deep uncertainty propagation framework for monocular/multi-view vision, coupled with Hierarchical Conformal Prediction (HCP), enabling statistically guaranteed uncertainty quantification in OCC for the first time. Innovatively, a KL-divergence-driven scoring function is introduced to significantly improve recall for long-tailed safety-critical classes (e.g., pedestrians, vehicles). By integrating multi-scale feature fusion and depth-guided geometric supervision, geometric completion and semantic segmentation mIoU improve by 11.58% and 12.95%, respectively; safety-class recall increases by 45%; average prediction set size shrinks by 92%; and further compression of 18% is achieved upon integrating Depth-UP.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Uncertainty Representations
• Computer Vision: Multi-modal Vision

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Security and Privacy: Large-scale security measurements
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

In the realm of autonomous vehicle perception, comprehending 3D scenes is paramount for tasks such as planning and mapping. Camera-based 3D Semantic Occupancy Prediction (OCC) aims to infer scene geometry and semantics from limited observations. While it has gained popularity due to affordability and rich visual cues, existing methods often neglect the inherent uncertainty in models. To address this, we propose an uncertainty-aware OCC method ($alpha$-OCC). We first introduce Depth-UP, an uncertainty propagation framework that improves geometry completion by up to 11.58% and semantic segmentation by up to 12.95% across various OCC models. For uncertainty quantification (UQ), we propose the hierarchical conformal prediction (HCP) method, effectively handling the high-level class imbalance in OCC datasets. On the geometry level, the novel KL-based score function significantly improves the occupied recall (45%) of safety-critical classes with minimal performance overhead (3.4% reduction). On UQ, our HCP achieves smaller prediction set sizes while maintaining the defined coverage guarantee. Compared with baselines, it reduces up to 92% set size, with 18% further reduction when integrated with Depth-UP. Our contributions advance OCC accuracy and robustness, marking a noteworthy step forward in autonomous perception systems.

Problem

• 3D Semantic Occupancy Prediction
• Uncertainty Handling
• Autonomous Driving Perception

Innovation

• 3D Semantic Occupancy Prediction
• Uncertainty Handling
• Class Imbalance Solution