Selective Prediction for Semantic Segmentation using Post-Hoc Confidence Estimation and Its Performance under Distribution Shift

2024-02-16

arXiv.org

AI Summary

Existing confidence scoring functions for binary semantic segmentation are misaligned with the Dice metric and require additional tuning data, limiting their reliability under distribution shifts. Method: We propose a training-free, image-level posterior confidence estimation method tailored for distribution shift scenarios. Our approach leverages feature-response statistics and uncertainty calibration to directly model the posterior reliability of predictions, enabling selective rejection of low-confidence outputs. Contribution/Results: We introduce a novel confidence measure explicitly aligned with the Dice optimization objective and establish a cross-domain generalization evaluation protocol. Evaluated on three low-resource medical imaging tasks, our method significantly improves selective prediction performance—achieving higher AUROC and F1-score at 90% coverage—while incurring zero training overhead and effectively mitigating performance degradation induced by distribution shift.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Computer Vision: Segmentation
• Reasoning under Uncertainty: Uncertainty Representations

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Security and Privacy: Large-scale security measurements

Abstract

Semantic segmentation plays a crucial role in various computer vision applications, yet its efficacy is often hindered by the lack of high-quality labeled data. To address this challenge, a common strategy is to leverage models trained on data from different populations, such as publicly available datasets. This approach, however, leads to the distribution shift problem, presenting a reduced performance on the population of interest. In scenarios where model errors can have significant consequences, selective prediction methods offer a means to mitigate risks and reduce reliance on expert supervision. This paper investigates selective prediction for semantic segmentation in low-resource settings, thus focusing on post-hoc confidence estimators applied to pre-trained models operating under distribution shift. We propose a novel image-level confidence measure tailored for semantic segmentation and demonstrate its effectiveness through experiments on three medical imaging tasks. Our findings show that post-hoc confidence estimators offer a cost-effective approach to reducing the impacts of distribution shift.

Problem

• Estimating reliable confidence scores for semantic segmentation
• Aligning confidence scores with Dice coefficient metric
• Avoiding additional data for tuning confidence estimators

Innovation

• Soft Dice Confidence aligns with Dice coefficient
• SDC requires no additional tuning data
• SDC outperforms prior confidence estimators