SUIT: Spatial-Spectral Union-Intersection Interaction Network for Hyperspectral Object Tracking

2025-08-10

AI Summary

Existing hyperspectral video tracking methods often neglect spectral-dimensional interactions, resulting in insufficient robustness under complex backgrounds and for small targets. To address this, we propose the first tracking network that explicitly models joint spatial-spectral interactions: (1) a spectral relation modeling module built upon set-theoretic union-intersection principles to capture inter-band spectral correlations; (2) a Transformer-based architecture to model long-range spatial dependencies across spectral bands; and (3) a spectral distribution alignment loss that enforces material consistency between template and predicted regions. Evaluated on multiple hyperspectral tracking benchmarks, our framework achieves state-of-the-art performance, demonstrating significant improvements in robustness against target deformation, appearance variation, and background clutter. The source code and pretrained models are publicly available.

Technology Category

• Computer Vision: Motion & Tracking
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for heterogeneous, signed, attributed, multi-relational, temporal, higher-order, and annotated Web-related graphs
• Web Mining and Content Analysis: Topic discovery and tracking
• Search and Retrieval-Augmented AI: Web query analysis, representation and understanding

Abstract

Hyperspectral videos (HSVs), with their inherent spatial-spectral-temporal structure, offer distinct advantages in challenging tracking scenarios such as cluttered backgrounds and small objects. However, existing methods primarily focus on spatial interactions between the template and search regions, often overlooking spectral interactions, leading to suboptimal performance. To address this issue, this paper investigates spectral interactions from both the architectural and training perspectives. At the architectural level, we first establish band-wise long-range spatial relationships between the template and search regions using Transformers. We then model spectral interactions using the inclusion-exclusion principle from set theory, treating them as the union of spatial interactions across all bands. This enables the effective integration of both shared and band-specific spatial cues. At the training level, we introduce a spectral loss to enforce material distribution alignment between the template and predicted regions, enhancing robustness to shape deformation and appearance variations. Extensive experiments demonstrate that our tracker achieves state-of-the-art tracking performance. The source code, trained models and results will be publicly available via https://github.com/bearshng/suit to support reproducibility.

Problem

• Addresses spectral interaction neglect in hyperspectral tracking
• Models spatial-spectral interactions using set theory principles
• Enhances robustness to deformation and appearance variations

Innovation

• Transformer-based spatial-spectral interaction modeling
• Spectral loss for material distribution alignment
• Union-intersection principle for band-specific cue integration