SMTrack: State-Aware Mamba for Efficient Temporal Modeling in Visual Tracking

2026-02-02

AI Summary

This work addresses the challenge of efficiently modeling long-range temporal dependencies in visual tracking under dynamic scenes, where existing approaches either rely on complex custom modules or incur high computational costs. To overcome these limitations, we propose a novel tracking framework based on the selective state space model (Mamba), which leverages a state-aware mechanism and inter-frame hidden state propagation to achieve concise yet effective long-term temporal modeling while maintaining linear computational complexity. Notably, our method requires no additional task-specific modules and achieves significant improvements in both accuracy and robustness across multiple benchmarks, thereby demonstrating the superiority and potential of state space models for visual tracking tasks.

Technology Category

• Computer Vision: Motion & Tracking
• Intelligent Robots: State Estimation
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Visual tracking aims to automatically estimate the state of a target object in a video sequence, which is challenging especially in dynamic scenarios. Thus, numerous methods are proposed to introduce temporal cues to enhance tracking robustness. However, conventional CNN and Transformer architectures exhibit inherent limitations in modeling long-range temporal dependencies in visual tracking, often necessitating either complex customized modules or substantial computational costs to integrate temporal cues. Inspired by the success of the state space model, we propose a novel temporal modeling paradigm for visual tracking, termed State-aware Mamba Tracker (SMTrack), providing a neat pipeline for training and tracking without needing customized modules or substantial computational costs to build long-range temporal dependencies. It enjoys several merits. First, we propose a novel selective state-aware space model with state-wise parameters to capture more diverse temporal cues for robust tracking. Second, SMTrack facilitates long-range temporal interactions with linear computational complexity during training. Third, SMTrack enables each frame to interact with previously tracked frames via hidden state propagation and updating, which releases computational costs of handling temporal cues during tracking. Extensive experimental results demonstrate that SMTrack achieves promising performance with low computational costs.

Problem

• visual tracking
• temporal modeling
• long-range dependencies
• computational efficiency
• state space model

Innovation

• State Space Model
• Temporal Modeling
• Visual Tracking
• Mamba
• Linear Complexity