A Framework for Multi-View Multiple Object Tracking using Single-View Multi-Object Trackers on Fish Data

2025-05-22

AI Summary

Single-view multi-object tracking (MOT) of small fish in underwater environments suffers from low accuracy and poor robustness due to complex 3D motion patterns and severe noise. Method: This paper proposes a novel multi-view collaborative tracking framework that tightly integrates FairMOT and YOLOv8, enabling joint high-precision detection, cross-view object association, stereo matching, and 3D trajectory reconstruction from synchronized binocular video sequences. The framework employs end-to-end joint optimization to eliminate error accumulation inherent in traditional pipeline-based approaches, while preserving model reusability. Contribution/Results: Experimental results demonstrate a 47% relative improvement in fish detection accuracy. To the best of our knowledge, this work establishes the first quantifiable analytical framework for 3D collective motion and inter-fish interaction patterns. Compared with single-view methods, it significantly enhances tracking robustness and behavioral interpretability.

Technology Category

• Computer Vision: Motion & Tracking
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Machine Learning: Multi-instance/Multi-view Learning

Application Category

• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Multi-object tracking (MOT) in computer vision has made significant advancements, yet tracking small fish in underwater environments presents unique challenges due to complex 3D motions and data noise. Traditional single-view MOT models often fall short in these settings. This thesis addresses these challenges by adapting state-of-the-art single-view MOT models, FairMOT and YOLOv8, for underwater fish detecting and tracking in ecological studies. The core contribution of this research is the development of a multi-view framework that utilizes stereo video inputs to enhance tracking accuracy and fish behavior pattern recognition. By integrating and evaluating these models on underwater fish video datasets, the study aims to demonstrate significant improvements in precision and reliability compared to single-view approaches. The proposed framework detects fish entities with a relative accuracy of 47% and employs stereo-matching techniques to produce a novel 3D output, providing a more comprehensive understanding of fish movements and interactions

Problem

• Adapting single-view MOT models for underwater fish tracking
• Developing multi-view framework to improve tracking accuracy
• Enhancing 3D fish movement analysis using stereo video

Innovation

• Multi-view framework using stereo video inputs
• Adapts FairMOT and YOLOv8 for underwater tracking
• Stereo-matching for novel 3D fish movement output