MeMoSORT: Memory-Assisted Filtering and Motion-Adaptive Association Metric for Multi-Person Tracking

2025-08-13

AI Summary

To address identity switches and track fragmentation caused by motion model mismatch and severe occlusion in multi-object tracking (MOT), this paper proposes a lightweight, online, real-time tracking method. The approach introduces two key innovations: (1) a Memory-enhanced Kalman Filter (MeKF) that compensates for motion model inaccuracies by incorporating historical state memory; and (2) a Motion-adaptive IoU (Mo-IoU) metric that dynamically expands the association range while suppressing false matches induced by high appearance similarity. Crucially, the method operates without re-identification modules, achieving an effective balance between accuracy and computational efficiency. Evaluated on DanceTrack and SportsMOT benchmarks, it achieves 67.9% and 82.1% HOTA, respectively—setting new state-of-the-art performance among online trackers.

Technology Category

• Computer Vision: Motion & Tracking
• Machine Learning: Mixture of Experts (MoE)
• Intelligent Robots: State Estimation

Application Category

• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Security and Privacy: Tracking, profiling, and countermeasures against them

Abstract

Multi-object tracking (MOT) in human-dominant scenarios, which involves continuously tracking multiple people within video sequences, remains a significant challenge in computer vision due to targets' complex motion and severe occlusions. Conventional tracking-by-detection methods are fundamentally limited by their reliance on Kalman filter (KF) and rigid Intersection over Union (IoU)-based association. The motion model in KF often mismatches real-world object dynamics, causing filtering errors, while rigid association struggles under occlusions, leading to identity switches or target loss. To address these issues, we propose MeMoSORT, a simple, online, and real-time MOT tracker with two key innovations. First, the Memory-assisted Kalman filter (MeKF) uses memory-augmented neural networks to compensate for mismatches between assumed and actual object motion. Second, the Motion-adaptive IoU (Mo-IoU) adaptively expands the matching space and incorporates height similarity to reduce the influence of detection errors and association failures, while remaining lightweight. Experiments on DanceTrack and SportsMOT show that MeMoSORT achieves state-of-the-art performance, with HOTA scores of 67.9% and 82.1%, respectively.

Problem

• Addresses limitations of Kalman filter in tracking motion
• Reduces identity switches caused by occlusions in MOT
• Improves association accuracy with adaptive IoU metric

Innovation

• Memory-assisted Kalman filter compensates motion mismatches
• Motion-adaptive IoU expands matching space adaptively
• Lightweight design ensures real-time multi-person tracking