A Multi-Annotator Study of Segmentation Noise and Uncertainty in Turbid Underwater Images

2026-08-15

AI Summary

This study addresses annotation uncertainty and noise in turbid underwater image segmentation through a large-scale multi-annotator investigation involving over one hundred participants. By systematically analyzing noise sources across varying turbidity levels via controlled experiments, privileged information assistance, and ensemble strategies, this work reveals systematic error patterns induced by turbidity in real-world underwater scenarios for the first time. Furthermore, it proposes effective methods to enhance annotation quality and releases an open-source dataset. This research bridges a critical gap in understanding annotation uncertainty within underwater segmentation, providing both theoretical foundations and practical guidelines for constructing high-quality underwater vision benchmarks.

Abstract

Label uncertainty and annotator disagreement are common challenges in the field of computer vision, yet their study has largely been confined to the medical domain or to generic image-recognition datasets. Underwater datasets are particularly susceptible to these issues due to the need for domain expertise, degraded visibility conditions, and the inherent difficulty of establishing reliable ground truth in inaccessible environments. Despite these challenges, annotation uncertainty in underwater imagery remains largely unexplored. In this work, we present the first systematic multi-annotator study of segmentation in real underwater scenes, with over 100 participants, and across varying, controlled levels of turbidity. We show that underwater datasets face many of the same annotation challenges as other vision tasks, while turbidity introduces additional systematic errors. We further investigate the main factors driving label noise and explore ways to improve annotation quality in turbid underwater environments, including privileged information, individual effort and annotator ensembles. All (meta-) data collected in this study will be available on the project page: https://vap.aau.dk/tubcertainty

Problem

• Underwater Image Segmentation
• Label Uncertainty
• Annotator Disagreement
• Turbidity
• Annotation Noise

Innovation

• Multi-Annotator Study
• Segmentation Uncertainty
• Turbid Underwater Images
• Label Noise
• Annotator Ensembles