Conformal Prediction Sets for Instance Segmentation

2026-02-10

AI Summary

This work addresses the lack of reliable uncertainty quantification in existing instance segmentation methods, which often fail to guarantee the proximity between predicted and ground-truth masks. To this end, it introduces conformal prediction—a first for instance segmentation—and proposes an algorithm with provable coverage guarantees. The method constructs, for any pixel coordinate, a confidence set of predictions that is theoretically guaranteed to contain at least one mask whose Intersection over Union (IoU) with the true instance mask exceeds a pre-specified threshold. By integrating both asymptotic and finite-sample theory, the approach achieves target coverage across diverse applications, including agricultural field delineation, cell segmentation, and vehicle detection. Moreover, it adaptively adjusts the size of the prediction set based on query difficulty and outperforms baseline methods such as Learn Then Test, conformal risk control, and morphological dilation.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Computer Vision: Segmentation
• Reasoning under Uncertainty: Other Foundations of Reasoning under Uncertainty

Application Category

• Security and Privacy: Large-scale security measurements
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Current instance segmentation models achieve high performance on average predictions, but lack principled uncertainty quantification: their outputs are not calibrated, and there is no guarantee that a predicted mask is close to the ground truth. To address this limitation, we introduce a conformal prediction algorithm to generate adaptive confidence sets for instance segmentation. Given an image and a pixel coordinate query, our algorithm generates a confidence set of instance predictions for that pixel, with a provable guarantee for the probability that at least one of the predictions has high Intersection-Over-Union (IoU) with the true object instance mask. We apply our algorithm to instance segmentation examples in agricultural field delineation, cell segmentation, and vehicle detection. Empirically, we find that our prediction sets vary in size based on query difficulty and attain the target coverage, outperforming existing baselines such as Learn Then Test, Conformal Risk Control, and morphological dilation-based methods. We provide versions of the algorithm with asymptotic and finite sample guarantees.

Problem

• instance segmentation
• uncertainty quantification
• conformal prediction
• calibration
• confidence sets

Innovation

• Conformal Prediction
• Instance Segmentation
• Uncertainty Quantification
• Confidence Sets
• Intersection-over-Union