Exploring Learning-based Motion Models in Multi-Object Tracking

2024-03-16

arXiv.org

AI Summary

To address the limitations of Kalman filters in modeling nonlinear motion and occlusions in multi-object tracking, this paper proposes MambaTrack—the first end-to-end online tracking framework built upon the state-space model Mamba—and its enhanced variant, MambaTrack+. The core contribution lies in pioneering the integration of Mamba for trajectory feature extraction, replacing handcrafted linear motion models with a data-driven approach capable of adaptively modeling complex trajectories (e.g., sharp turns, long-term occlusions). Additionally, we introduce a trajectory feature adaptive extraction mechanism and a lightweight online update architecture. On DanceTrack, MambaTrack achieves 56.1 HOTA and 54.9 IDF1, setting a new state-of-the-art at the time. On SportsMOT, it comprehensively outperforms all motion-model-based trackers, demonstrating both the effectiveness and generalizability of nonlinear motion modeling in tracking.

Technology Category

• Computer Vision: Motion & Tracking
• Intelligent Robots: State Estimation
• Machine Learning: Mixture of Experts (MoE)

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

In the field of multi-object tracking (MOT), traditional methods often rely on the Kalman Filter for motion prediction, leveraging its strengths in linear motion scenarios. However, the inherent limitations of these methods become evident when confronted with complex, nonlinear motions and occlusions prevalent in dynamic environments like sports and dance. This paper explores the possibilities of replacing the Kalman Filter with various learning-based motion model that effectively enhances tracking accuracy and adaptability beyond the constraints of Kalman Filter-based systems. In this paper, we proposed MambaTrack, an online motion-based tracker that outperforms all existing motion-based trackers on the challenging DanceTrack and SportsMOT datasets. Moreover, we further exploit the potential of the state-space-model in trajectory feature extraction to boost the tracking performance and proposed MambaTrack+, which achieves the state-of-the-art performance on DanceTrack dataset with 56.1 HOTA and 54.9 IDF1.

Problem

• Multi-Object-Tracking
• Complex-Motion-Prediction
• Kalman-Filter-Limitations

Innovation

• State Space Models
• Nonlinear Motion
• Occlusion Handling