Motion-Aware Transformer for Multi-Object Tracking

2025-09-25

AI Summary

In complex motion and dense-scene multi-object tracking (MOT), coupling between detection and tracking queries leads to query conflicts and inaccurate association. To address this, we propose the Motion-Aware Transformer (MATR), the first end-to-end Transformer framework that explicitly models inter-frame object motion. MATR introduces motion-predictive trajectory queries updated prior to association, thereby decoupling detection and tracking queries. This design alleviates query interference, improves detection-association consistency, and enhances training stability. Built upon the DETR architecture, MATR integrates temporal motion cues within the decoder to refine trajectory queries. Experiments demonstrate state-of-the-art performance without external data: MATR achieves a HOTA of 71.3 (+9.1) on DanceTrack, and sets new SOTA results on SportsMOT and BDD100K.

Technology Category

• Computer Vision: Motion & Tracking
• Intelligent Robots: Motion and Path Planning
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Security and Privacy: Tracking, profiling, and countermeasures against them

Abstract

Multi-object tracking (MOT) in videos remains challenging due to complex object motions and crowded scenes. Recent DETR-based frameworks offer end-to-end solutions but typically process detection and tracking queries jointly within a single Transformer Decoder layer, leading to conflicts and degraded association accuracy. We introduce the Motion-Aware Transformer (MATR), which explicitly predicts object movements across frames to update track queries in advance. By reducing query collisions, MATR enables more consistent training and improves both detection and association. Extensive experiments on DanceTrack, SportsMOT, and BDD100k show that MATR delivers significant gains across standard metrics. On DanceTrack, MATR improves HOTA by more than 9 points over MOTR without additional data and reaches a new state-of-the-art score of 71.3 with supplementary data. MATR also achieves state-of-the-art results on SportsMOT (72.2 HOTA) and BDD100k (54.7 mTETA, 41.6 mHOTA) without relying on external datasets. These results demonstrate that explicitly modeling motion within end-to-end Transformers offers a simple yet highly effective approach to advancing multi-object tracking.

Problem

• Addresses complex object motions in crowded video tracking
• Resolves detection and tracking query conflicts in Transformers
• Improves multi-object tracking accuracy through motion prediction

Innovation

• Motion-Aware Transformer predicts object movements across frames
• Explicit motion modeling reduces query conflicts in tracking
• Transformer-based end-to-end framework improves detection and association