MUOT_3M: A 3 Million Frame Multimodal Underwater Benchmark and the MUTrack Tracking Method

2026-02-20

AI Summary

Underwater object tracking is hindered by the scarcity of large-scale, multimodal datasets; existing benchmarks are limited in scale and restricted to RGB modality, making them ill-equipped to handle challenges such as color distortion, turbidity, and low visibility. To address this, this work introduces MUOT_3M, the first pseudo-multimodal underwater tracking benchmark comprising three million frames, which incorporates enhanced RGB, estimated depth, and language modalities. The authors further propose MUTrack, a SAM-based method that efficiently transfers multimodal knowledge to a unimodal model through visual-geometric alignment, vision-language fusion, and a four-stage knowledge distillation framework. Evaluated across five underwater benchmarks, MUTrack achieves up to an 8.40% improvement in AUC and a 7.80% gain in precision over the strongest state-of-the-art method, while operating at 24 FPS.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Multimodal Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Underwater Object Tracking (UOT) is crucial for efficient marine robotics, large scale ecological monitoring, and ocean exploration; however, progress has been hindered by the scarcity of large, multimodal, and diverse datasets. Existing benchmarks remain small and RGB only, limiting robustness under severe color distortion, turbidity, and low visibility conditions. We introduce MUOT_3M, the first pseudo multimodal UOT benchmark comprising 3 million frames from 3,030 videos (27.8h) annotated with 32 tracking attributes, 677 fine grained classes, and synchronized RGB, estimated enhanced RGB, estimated depth, and language modalities validated by a marine biologist. Building upon MUOT_3M, we propose MUTrack, a SAM-based multimodal to unimodal tracker featuring visual geometric alignment, vision language fusion, and four level knowledge distillation that transfers multimodal knowledge into a unimodal student model. Extensive evaluations across five UOT benchmarks demonstrate that MUTrack achieves up to 8.40% higher AUC and 7.80% higher precision than the strongest SOTA baselines while running at 24 FPS. MUOT_3M and MUTrack establish a new foundation for scalable, multimodally trained yet practically deployable underwater tracking.

Problem

• Underwater Object Tracking
• multimodal benchmark
• dataset scarcity
• color distortion
• low visibility

Innovation

• multimodal underwater tracking
• knowledge distillation
• vision-language fusion
• SAM-based tracker
• underwater benchmark