FusionSORT: Fusion Methods for Online Multi-object Visual Tracking

2025-01-01

AI Summary

This paper addresses the data association challenge in online multi-object tracking, arising from severe target occlusion, visual similarity, and dynamically varying trajectory reliability. We systematically investigate fusion strategies for four heterogeneous cues: motion (Kalman filter prediction), appearance features, high IoU scores, and trajectory confidence. For the first time, we comparatively analyze four fusion paradigms—minimum fusion, weighted IoU summation, Kalman gating, and Hadamard product-based cost fusion—and characterize their distinct impacts on IDF1 and MOTA. Extensive evaluation on MOT17, MOT20, and DanceTrack demonstrates that Hadamard fusion significantly enhances matching robustness, yielding up to a 3.2% IDF1 improvement. The study establishes a reproducible methodological framework for cue fusion, providing a practical benchmark for designing lightweight yet highly robust trackers.

Technology Category

• Computer Vision: Motion & Tracking
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Machine Learning: Multi-instance/Multi-view Learning

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Web Mining and Content Analysis: Topic discovery and tracking

Abstract

In this work, we investigate four different fusion methods for associating detections to tracklets in multi-object visual tracking. In addition to considering strong cues such as motion and appearance information, we also consider weak cues such as height intersection-over-union (height-IoU) and tracklet confidence information in the data association using different fusion methods. These fusion methods include minimum, weighted sum based on IoU, Kalman filter (KF) gating, and hadamard product of costs due to the different cues. We conduct extensive evaluations on validation sets of MOT17, MOT20 and DanceTrack datasets, and find out that the choice of a fusion method is key for data association in multi-object visual tracking. We hope that this investigative work helps the computer vision research community to use the right fusion method for data association in multi-object visual tracking.

Problem

• Multi-Object Tracking
• Feature Fusion
• Accuracy Improvement

Innovation

• FusionSORT
• Multi-Object Tracking
• Integrated Strategies