A Comparative Study on Multi-task Uncertainty Quantification in Semantic Segmentation and Monocular Depth Estimation

2024-05-27

AI Summary

This work addresses uncertainty quantification in multi-task learning—specifically joint semantic segmentation and monocular depth estimation—for safety-critical applications such as autonomous driving, with emphasis on improving out-of-distribution (OOD) robustness and predictive calibration. We systematically compare three uncertainty estimation paradigms: Monte Carlo Dropout, deep sub-ensembles, and deep ensembles, and propose a median-based adaptive pixel-wise confidence thresholding strategy. Experimental results demonstrate that deep ensembles significantly outperform alternatives in both uncertainty calibration and OOD detection. Moreover, multi-task co-modeling jointly enhances the quality of uncertainty estimates for both tasks. The median threshold exhibits strong robustness across diverse domains and datasets, making it a reliable default choice. Our study establishes a reproducible benchmark and practical methodology for uncertainty quantification in multi-task vision systems.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Uncertainty Representations
• Computer Vision: Multi-modal Vision

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Deep neural networks excel in perception tasks such as semantic segmentation and monocular depth estimation, making them indispensable in safety-critical applications like autonomous driving and industrial inspection. However, they often suffer from overconfidence and poor explainability, especially for out-of-domain data. While uncertainty quantification has emerged as a promising solution to these challenges, multi-task settings have yet to be explored. In an effort to shed light on this, we evaluate Monte Carlo Dropout, Deep Sub-Ensembles, and Deep Ensembles for joint semantic segmentation and monocular depth estimation. Thereby, we reveal that Deep Ensembles stand out as the preferred choice, particularly in out-of-domain scenarios, and show the potential benefit of multi-task learning with regard to the uncertainty quality in comparison to solving both tasks separately. Additionally, we highlight the impact of employing different uncertainty thresholds to classify pixels as certain or uncertain, with the median uncertainty emerging as a robust default.

Problem

• Multi-task Learning
• Deep Neural Networks
• Uncertainty Estimation

Innovation

• Multi-task Learning
• Uncertainty Estimation
• Robustness Improvement