SpecTrack: Spectral Prompt Guided Adaptive Experts for Multispectral Object Tracking

2026-07-07

AI Summary

Existing multispectral object tracking methods employ fixed-capacity spectral-spatial processing pipelines, which struggle to adapt to varying levels of difficulty across frames and target states. This work proposes SpecTrack, formulating tracking as a dynamic model capacity allocation problem conditioned on the search region. It introduces a novel Spectral Adaptive Mixture-of-Experts (SAMoE) module coupled with a spectral prompt router that enables sparse expert activation based on semantic, boundary, and channel-wise variation cues, while shared global experts enhance contextual consistency. The method achieves state-of-the-art or competitive AUC scores on MUST (65.2%), MSITrack (51.9%), and HOTC20 (72.6%). Notably, the lightweight variant SpecTrack-B224 attains 62.4% AUC on MUST at 43.7 FPS and achieves an AO of 79.3% on GOT-10k, demonstrating both efficiency and strong generalization capability.

Abstract

Multispectral image(MSI) and hyperspectral image(HSI) object tracking object tracking exploits recorded band-wise observations to improve target--background discrimination under similar RGB appearance, mixed pixels, illumination variation, occlusion, and clutter. However, existing trackers commonly process all search regions through a fixed capacity spectral--spatial path, ignoring that tracking difficulty varies substantially across frames and target states. Clear regions may require only lightweight local discrimination, whereas ambiguous boundaries and spectrally similar distractors often demand stronger contextual reasoning. To address this limitation, we propose SpecTrack, a spectral--spatial complexity-aware tracker that formulates MSI tracking as search-region-level adaptive capacity allocation. Its core component, the Spectral Adaptive Mixture-of-Experts (SAMoE) module, provides a capacity-ordered expert pool with progressively increasing latent rank, receptive field, and depth. Expert selection is guided by a Spectral Prompt Router, which fuses semantic context, spatial boundary cues, and a latent channel-variation cue computed after multispectral patch embedding to activate a sparse subset of SAMoE experts for each search region. In parallel, a Shared Global Expert supplies common latent spectral--spatial context to reduce fragmented sparse-routing decisions. Experiments on MUST, MSITrack, and HOTC20 demonstrate a favorable accuracy--efficiency trade-off. The accuracy-oriented SpecTrack-L384 achieves state-of-the-art or highly competitive AUCs of 65.2\%, 51.9\%, and 72.6\% on the three benchmarks, while the balanced SpecTrack-B224 reaches 62.4\% AUC at 43.7 FPS on MUST. An additional GOT-10k evaluation indicates RGB-domain architectural generalization, with SpecTrack-L384 achieving 79.3\% AO.

Problem

• multispectral object tracking
• adaptive capacity allocation
• spectral-spatial complexity
• tracking difficulty variation
• search region adaptation

Innovation

• adaptive mixture-of-experts
• spectral prompt routing
• multispectral object tracking
• complexity-aware tracking
• sparse expert activation