AquaMonitor: A multimodal multi-view image sequence dataset for real-life aquatic invertebrate biodiversity monitoring

2025-05-28

AI Summary

Existing aquatic invertebrate vision datasets lack standardized acquisition protocols and ecological validity, limiting their utility for field-based biodiversity assessment. Method: We introduce the first large-scale, multimodal, multi-view image sequence dataset tailored for ecological monitoring—comprising 43,189 specimens and 2.7 million images—integrated with DNA barcodes, dry mass, and morphometric measurements. We establish the first end-to-end standardized pipeline spanning field collection, laboratory imaging, and cross-modal alignment, and propose the Monitoring benchmark, which jointly addresses open-set recognition, distribution shift, and extreme class imbalance. Contribution/Results: Our approach integrates multi-view modeling, fine-grained classification, and distributionally robust learning. We release the largest computer vision dataset for aquatic invertebrates to date, alongside three rigorously designed benchmarks and strong baseline models. Performance gains on the Monitoring benchmark directly support statutory biological water quality monitoring practices.

Technology Category

• Computer Vision: Multi-modal Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Machine Learning: Multimodal Learning

Application Category

• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Security and Privacy: Data transparency and provenance
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data

Abstract

This paper presents the AquaMonitor dataset, the first large computer vision dataset of aquatic invertebrates collected during routine environmental monitoring. While several large species identification datasets exist, they are rarely collected using standardized collection protocols, and none focus on aquatic invertebrates, which are particularly laborious to collect. For AquaMonitor, we imaged all specimens from two years of monitoring whenever imaging was possible given practical limitations. The dataset enables the evaluation of automated identification methods for real-life monitoring purposes using a realistically challenging and unbiased setup. The dataset has 2.7M images from 43,189 specimens, DNA sequences for 1358 specimens, and dry mass and size measurements for 1494 specimens, making it also one of the largest biological multi-view and multimodal datasets to date. We define three benchmark tasks and provide strong baselines for these: 1) Monitoring benchmark, reflecting real-life deployment challenges such as open-set recognition, distribution shift, and extreme class imbalance, 2) Classification benchmark, which follows a standard fine-grained visual categorization setup, and 3) Few-shot benchmark, which targets classes with only few training examples from very fine-grained categories. Advancements on the Monitoring benchmark can directly translate to improvement of aquatic biodiversity monitoring, which is an important component of regular legislative water quality assessment in many countries.

Problem

• Develops AquaMonitor dataset for aquatic invertebrate biodiversity monitoring
• Addresses real-life challenges in automated species identification methods
• Provides benchmarks for monitoring, classification, and few-shot learning tasks

Innovation

• Multimodal dataset with images, DNA, and measurements
• Standardized real-life monitoring protocol implementation
• Benchmark tasks for open-set and few-shot challenges