When Trackers Date Fish: A Benchmark and Framework for Underwater Multiple Fish Tracking

2025-07-08

AI Summary

Underwater multi-fish tracking remains challenging due to severe occlusions, high inter-class appearance similarity, and nonlinear motion—further hindered by the absence of dedicated benchmarks and tailored methods. To address this, we introduce MFT25, the first benchmark for real-world complex underwater multi-fish tracking, comprising 15 videos and 48,066 frames with fine-grained annotations. We propose SU-T, an end-to-end tracking framework specifically designed for underwater fish dynamics: (i) an Unscented Kalman Filter (UKF)-based state predictor modeling nonlinear swimming behavior; (ii) a scale-aware detection module enhancing robustness to size variation; and (iii) FishIoU, a novel association metric grounded in morphological modeling of fish shape. Evaluated on MFT25, SU-T achieves 34.1 HOTA and 44.6 IDF1—substantially outperforming prior methods. This work establishes the first open-source underwater multi-fish tracking dataset, introduces a new methodological paradigm, and sets a new state-of-the-art baseline.

Technology Category

• Computer Vision: Motion & Tracking
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Reasoning under Uncertainty: Uncertainty Representations

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Web Mining and Content Analysis: Topic discovery and tracking

Abstract

Multiple object tracking (MOT) technology has made significant progress in terrestrial applications, but underwater tracking scenarios remain underexplored despite their importance to marine ecology and aquaculture. We present Multiple Fish Tracking Dataset 2025 (MFT25), the first comprehensive dataset specifically designed for underwater multiple fish tracking, featuring 15 diverse video sequences with 408,578 meticulously annotated bounding boxes across 48,066 frames. Our dataset captures various underwater environments, fish species, and challenging conditions including occlusions, similar appearances, and erratic motion patterns. Additionally, we introduce Scale-aware and Unscented Tracker (SU-T), a specialized tracking framework featuring an Unscented Kalman Filter (UKF) optimized for non-linear fish swimming patterns and a novel Fish-Intersection-over-Union (FishIoU) matching that accounts for the unique morphological characteristics of aquatic species. Extensive experiments demonstrate that our SU-T baseline achieves state-of-the-art performance on MFT25, with 34.1 HOTA and 44.6 IDF1, while revealing fundamental differences between fish tracking and terrestrial object tracking scenarios. MFT25 establishes a robust foundation for advancing research in underwater tracking systems with important applications in marine biology, aquaculture monitoring, and ecological conservation. The dataset and codes are released at https://vranlee.github.io/SU-T/.

Problem

• Underwater multiple fish tracking lacks comprehensive datasets and benchmarks
• Existing tracking methods struggle with fish occlusions and erratic motion
• Need specialized frameworks for non-linear fish movement and morphology

Innovation

• First underwater fish tracking dataset MFT25
• Scale-aware Unscented Tracker for fish patterns
• FishIoU matching for aquatic species morphology