Benchmarking Out-of-Distribution Detection for Plankton Recognition: A Systematic Evaluation of Advanced Methods in Marine Ecological Monitoring

2025-10-20

AI Summary

Plankton identification models suffer from reliability degradation in real-world deployment due to out-of-distribution (OoD) shifts between training and test data—stemming from high morphological complexity, extreme species diversity, and continual discovery of novel taxa. To address this, we introduce DYB-PlanktonNet, the first large-scale OoD detection benchmark specifically designed for plankton recognition, encompassing both near-OoD and far-OoD scenarios. We systematically evaluate 22 state-of-the-art vision-based anomaly detection methods—including ViM, MSP, and ODIN—under standardized protocols. Experimental results demonstrate that ViM achieves superior performance across all metrics, exhibiting exceptional robustness—particularly under far-OoD conditions. This work bridges a critical gap in the field by establishing the first unified benchmark and comprehensive evaluation framework for OoD detection in plankton analysis, thereby enabling more trustworthy automated marine monitoring systems.

Technology Category

• Data Mining & Knowledge Management: Anomaly/Outlier Detection
• Computer Vision: Object Detection & Categorization
• Machine Learning: Multi-class/Multi-label Learning & Extreme Classification

Application Category

• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Automated plankton recognition models face significant challenges during real-world deployment due to distribution shifts (Out-of-Distribution, OoD) between training and test data. This stems from plankton's complex morphologies, vast species diversity, and the continuous discovery of novel species, which leads to unpredictable errors during inference. Despite rapid advancements in OoD detection methods in recent years, the field of plankton recognition still lacks a systematic integration of the latest computer vision developments and a unified benchmark for large-scale evaluation. To address this, this paper meticulously designed a series of OoD benchmarks simulating various distribution shift scenarios based on the DYB-PlanktonNet dataset cite{875n-f104-21}, and systematically evaluated twenty-two OoD detection methods. Extensive experimental results demonstrate that the ViM cite{wang2022vim} method significantly outperforms other approaches in our constructed benchmarks, particularly excelling in Far-OoD scenarios with substantial improvements in key metrics. This comprehensive evaluation not only provides a reliable reference for algorithm selection in automated plankton recognition but also lays a solid foundation for future research in plankton OoD detection. To our knowledge, this study marks the first large-scale, systematic evaluation and analysis of Out-of-Distribution data detection methods in plankton recognition. Code is available at https://github.com/BlackJack0083/PlanktonOoD.

Problem

• Evaluating OoD detection methods for plankton recognition
• Addressing distribution shifts in marine ecological monitoring
• Systematic benchmarking of advanced computer vision techniques

Innovation

• Systematically evaluated 22 OOD detection methods
• Designed benchmarks simulating distribution shift scenarios
• Identified ViM method outperforming others significantly