Probabilistic Trajectory GOSPA: A Metric for Uncertainty-Aware Multi-Object Tracking Performance Evaluation

2025-06-18

AI Summary

This paper addresses the neglect of both trajectory existence uncertainty and state estimation uncertainty in multi-object tracking (MOT) evaluation, proposing the first unified trajectory-level performance metric that jointly models both. Methodologically, it extends the Generalized Optimal Subpattern Assignment (GOSPA) metric to the trajectory domain and integrates probabilistic outputs to jointly quantify existence and state uncertainties; it employs multidimensional assignment modeling with linear programming relaxation to ensure polynomial-time computability and full adherence to metric axioms. Contributions include: (i) an interpretable error decomposition framework—covering localization, existence probability mismatch, missed detections/false alarms, and identity switches; (ii) significantly enhanced sensitivity to tracker uncertainty; and (iii) empirical validation showing principled error decomposition and high consistency with human assessment, outperforming conventional deterministic evaluation methods.

Technology Category

• Computer Vision: Motion & Tracking
• Reasoning under Uncertainty: Uncertainty Representations
• Machine Learning: Calibration & Uncertainty Quantification

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• User Modeling, Personalization and Recommendation: Metrics for user behavior and evaluating success
• Security and Privacy: Tracking, profiling, and countermeasures against them

Abstract

This paper presents a generalization of the trajectory general optimal sub-pattern assignment (GOSPA) metric for evaluating multi-object tracking algorithms that provide trajectory estimates with track-level uncertainties. This metric builds on the recently introduced probabilistic GOSPA metric to account for both the existence and state estimation uncertainties of individual object states. Similar to trajectory GOSPA (TGOSPA), it can be formulated as a multidimensional assignment problem, and its linear programming relaxation--also a valid metric--is computable in polynomial time. Additionally, this metric retains the interpretability of TGOSPA, and we show that its decomposition yields intuitive costs terms associated to expected localization error and existence probability mismatch error for properly detected objects, expected missed and false detection error, and track switch error. The effectiveness of the proposed metric is demonstrated through a simulation study.

Problem

• Extends trajectory GOSPA for uncertainty-aware tracking evaluation
• Formulates multidimensional assignment with polynomial-time computation
• Decomposes into intuitive error costs for performance analysis

Innovation

• Generalizes trajectory GOSPA for uncertainty-aware tracking
• Formulates as polynomial-time solvable assignment problem
• Decomposes into interpretable error cost terms