The Coralscapes Dataset: Semantic Scene Understanding in Coral Reefs

2025-03-25

AI Summary

Coral reefs worldwide are undergoing rapid degradation due to climate change and local anthropogenic stressors, necessitating high-resolution, scalable automated monitoring. Existing coral image analysis is hampered by limited, low-quality annotated datasets. To address this, we introduce CoralSeg—the first general-purpose, densely annotated semantic segmentation benchmark for coral reef ecology—comprising 2,075 high-resolution underwater images, 39 benthic organism classes, and 174,000 expert-validated pixel-level masks, uniquely aligned with the Cityscapes annotation schema. Leveraging CoralSeg, we conduct a systematic evaluation of state-of-the-art architectures—including U-Net, DeepLabv3+, and Mask2Former—and demonstrate the efficacy of cross-domain transfer learning for marine vision tasks. Our method achieves SOTA performance across multiple small-scale coral datasets, significantly improving live-coral detection accuracy and cross-dataset generalization. CoralSeg establishes a standardized visual foundation for autonomous underwater ecological monitoring and robotic applications.

Technology Category

• Computer Vision: Segmentation
• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Vertical and domain-specific search
• Semantics and Knowledge: Scalable techniques for the creation, curation, publication, maintenance, and consumption of large, Web-based, structured, reusable, knowledge graphs and ontologies
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

Coral reefs are declining worldwide due to climate change and local stressors. To inform effective conservation or restoration, monitoring at the highest possible spatial and temporal resolution is necessary. Conventional coral reef surveying methods are limited in scalability due to their reliance on expert labor time, motivating the use of computer vision tools to automate the identification and abundance estimation of live corals from images. However, the design and evaluation of such tools has been impeded by the lack of large high quality datasets. We release the Coralscapes dataset, the first general-purpose dense semantic segmentation dataset for coral reefs, covering 2075 images, 39 benthic classes, and 174k segmentation masks annotated by experts. Coralscapes has a similar scope and the same structure as the widely used Cityscapes dataset for urban scene segmentation, allowing benchmarking of semantic segmentation models in a new challenging domain which requires expert knowledge to annotate. We benchmark a wide range of semantic segmentation models, and find that transfer learning from Coralscapes to existing smaller datasets consistently leads to state-of-the-art performance. Coralscapes will catalyze research on efficient, scalable, and standardized coral reef surveying methods based on computer vision, and holds the potential to streamline the development of underwater ecological robotics.

Problem

• Automating coral identification and abundance estimation from images
• Addressing lack of large high-quality coral reef datasets
• Enabling scalable coral reef monitoring via computer vision

Innovation

• First dense semantic segmentation dataset for coral reefs
• Benchmarking models for coral reef semantic segmentation
• Transfer learning improves performance on smaller datasets