WildFin: An In-the-Wild Dataset for Fish Behavioral Recognition

2026-08-21

AI Summary

为解决野外视频数据标注成本高及现有模型在复杂海洋环境中的失效问题，本文通过创建WildFin数据集并使用计算机视觉方法进行鱼类行为识别。

Abstract

Recent advances in field technology have led to a massive influx of in-the-wild video data for ecological science. The primary bottleneck in leveraging this data is the high cost of expert annotation. While computer vision offers a potential solution, current models frequently fail when deployed in complex marine environments. To characterize these failures, we introduce WildFin, a novel benchmark for fish behavior recognition collected and annotated by ecologists.WildFin spans two critical real-world paradigms: stationary cameras monitoring groups of fish and dynamic divers following individual subjects. The dataset represents a massive curation effort, involving 1,350 hours of fieldwork and 600 hours of expert annotation to produce 9 hours of behavioral data with over 2 million frame-by-frame labels. We benchmark modern vision foundation models and quantify tradeoffs between static and spatiotemporal architectures, revealing the substantial gap that remains between current model capabilities and the demands of real-world underwater behavioral analysis. Project website: https://team-wildfin.github.io/.

Problem

• fish behavior recognition
• in-the-wild video data
• computer vision
• marine environments
• expert annotation

Innovation

• WildFin
• fish behavior recognition
• in-the-wild dataset
• spatiotemporal architectures