MUST: The First Dataset and Unified Framework for Multispectral UAV Single Object Tracking

2025-03-22

AI Summary

To address the limitations of RGB-based trackers in single-object UAV tracking—particularly under severe occlusion and small target scales—this paper introduces MUST, the first large-scale multispectral UAV tracking benchmark comprising 250 sequences, thereby filling a critical gap in evaluation resources. We propose UNTrack, a novel framework that jointly models spectral, spatial, and temporal cues: it employs an asymmetric Transformer architecture integrated with a spectral background suppression mechanism and a dynamically updated spectral prompt encoder to enable adaptive cross-modal feature fusion. Evaluated on MUST, UNTrack significantly outperforms both RGB-only and state-of-the-art multispectral trackers in accuracy while maintaining superior computational efficiency. To foster reproducibility and community advancement, both the codebase and the MUST dataset are publicly released.

Technology Category

• Computer Vision: Motion & Tracking
• Machine Learning: Multimodal Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Web Mining and Content Analysis: Topic discovery and tracking
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

UAV tracking faces significant challenges in real-world scenarios, such as small-size targets and occlusions, which limit the performance of RGB-based trackers. Multispectral images (MSI), which capture additional spectral information, offer a promising solution to these challenges. However, progress in this field has been hindered by the lack of relevant datasets. To address this gap, we introduce the first large-scale Multispectral UAV Single Object Tracking dataset (MUST), which includes 250 video sequences spanning diverse environments and challenges, providing a comprehensive data foundation for multispectral UAV tracking. We also propose a novel tracking framework, UNTrack, which encodes unified spectral, spatial, and temporal features from spectrum prompts, initial templates, and sequential searches. UNTrack employs an asymmetric transformer with a spectral background eliminate mechanism for optimal relationship modeling and an encoder that continuously updates the spectrum prompt to refine tracking, improving both accuracy and efficiency. Extensive experiments show that our proposed UNTrack outperforms state-of-the-art UAV trackers. We believe our dataset and framework will drive future research in this area. The dataset is available on https://github.com/q2479036243/MUST-Multispectral-UAV-Single-Object-Tracking.

Problem

• Addresses lack of multispectral UAV tracking datasets
• Proposes unified framework for spectral-spatial-temporal feature encoding
• Improves tracking accuracy in small-target occlusion scenarios

Innovation

• Introduces MUST: first multispectral UAV tracking dataset
• Proposes UNTrack: unified spectral-spatial-temporal feature framework
• Employs asymmetric transformer with spectral background elimination