BenthiCat: An opti-acoustic dataset for advancing benthic classification and habitat mapping

2025-10-06

AI Summary

To address the critical bottlenecks of scarce large-scale labeled data and insufficient cross-modal alignment in seafloor habitat mapping, this work introduces CatHab—the first million-scale, AUV-collected multimodal seafloor habitat dataset, acquired along the Catalan coast of Spain. CatHab synchronously provides co-registered sidescan sonar imagery, bathymetric maps, and underwater optical images, accompanied by ~36,000 sonar patches with pixel-level segmentation masks and full raw sensor data. We establish a standardized preprocessing pipeline and release an open-source annotation toolkit. Crucially, we introduce the first acoustic–optical cross-modal alignment benchmark, enabling self-supervised representation learning and end-to-end habitat classification. CatHab significantly advances research in multisensor fusion modeling and autonomous seafloor habitat identification, offering a foundational resource for benchmarking and developing robust, modality-robust habitat perception systems.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Computer Vision: Multi-modal Vision
• Machine Learning: Multimodal Learning

Application Category

• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

Benthic habitat mapping is fundamental for understanding marine ecosystems, guiding conservation efforts, and supporting sustainable resource management. Yet, the scarcity of large, annotated datasets limits the development and benchmarking of machine learning models in this domain. This paper introduces a thorough multi-modal dataset, comprising about a million side-scan sonar (SSS) tiles collected along the coast of Catalonia (Spain), complemented by bathymetric maps and a set of co-registered optical images from targeted surveys using an autonomous underwater vehicle (AUV). Approximately um{36000} of the SSS tiles have been manually annotated with segmentation masks to enable supervised fine-tuning of classification models. All the raw sensor data, together with mosaics, are also released to support further exploration and algorithm development. To address challenges in multi-sensor data fusion for AUVs, we spatially associate optical images with corresponding SSS tiles, facilitating self-supervised, cross-modal representation learning. Accompanying open-source preprocessing and annotation tools are provided to enhance accessibility and encourage research. This resource aims to establish a standardized benchmark for underwater habitat mapping, promoting advancements in autonomous seafloor classification and multi-sensor integration.

Problem

• Addressing scarcity of annotated datasets for benthic habitat mapping
• Providing multi-modal sonar and optical data for marine classification
• Establishing benchmark for underwater habitat mapping and sensor fusion

Innovation

• Multi-modal dataset with sonar and optical images
• Spatial association for cross-modal representation learning
• Open-source preprocessing and annotation tools provided