Why Domain Matters: Domain-Aware Benchmarking of Underwater Object Detection and Annotation Quality

2026-07-12

AI Summary

Underwater object detection suffers severely from domain shift, yet existing evaluations predominantly rely on synthetic data that fails to reflect real-world performance. This work proposes the first realistic domain labeling framework grounded in appearance, scene composition, and acquisition geometry, establishing a domain-aware evaluation protocol to systematically quantify the impact of domain factors on both human annotation quality and detector performance. Through domain-specific modeling and cross-domain assessment, the study reveals substantial performance gaps and annotation discrepancies across distinct underwater domains, offering critical insights for optimizing data collection strategies, refining annotation protocols, and designing robust detectors resilient to domain variations.

Abstract

Underwater object detection is strongly affected by domain shift, where performance can vary significantly across different locations, habitats, and deployment conditions. However, detector performance is typically evaluated using aggregate metrics that hide failures in specific environments, while existing domain generalization benchmarks often rely on synthetic variations that do not reflect real-world conditions. We introduce a framework that characterizes underwater images by appearance, scene composition, and acquisition geometry to assign domain labels. Using this framework, we perform the first systematic study of how domain factors influence both human annotation quality in underwater object detection datasets and deep learning-based detector performance, revealing substantial domain-dependent discrepancies. By incorporating physically meaningful domain labels, domain shift becomes something we can characterize, measure, benchmark, and act on. We highlight how this can be used to guide data collection and annotation, design more informative benchmarks, and assess detector robustness across diverse underwater environments.

Problem

• domain shift
• underwater object detection
• annotation quality
• benchmarking
• domain-aware

Innovation

• domain-aware benchmarking
• underwater object detection
• domain shift
• annotation quality
• domain characterization