SSP-GNN: Learning to Track via Bilevel Optimization

2024-07-05

Fusion

AI Summary

Multi-object tracking (MOT) faces challenges in jointly leveraging kinematic and re-identification features, and conventional trajectory inference methods are non-differentiable, hindering end-to-end learning. Method: We propose an end-to-end differentiable MOT framework based on graph neural networks (GNNs). It constructs a temporal tracking graph integrating kinematic and appearance features, employs the differentiable shortest path (SSP) algorithm for trajectory association, and—novelly for graph-based MOT—adopts bilevel optimization to jointly learn GNN parameters and SSP solutions. A dedicated alignment loss explicitly guides SSP outputs toward ground-truth trajectories. Results: Extensive experiments on diverse complex synthetic scenarios demonstrate significant improvements over strong baselines. The method exhibits superior robustness under detection noise, occlusion, and variations in hyperparameters, validating its generalizability and reliability.

Technology Category

• Computer Vision: Motion & Tracking
• Search and Optimization: Learning to Search
• Machine Learning: Graph-based Machine Learning

Application Category

• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

We propose a graph-based tracking formulation for multi-object tracking (MOT) where target detections contain kinematic information and re-identification features (attributes). Our method applies a successive shortest paths (SSP) algorithm to a tracking graph defined over a batch of frames. The edge costs in this tracking graph are computed via message-passing network, a graph neural network (GNN) variant. The parameters of the GNN, and hence, the tracker, are learned end-to-end on a training set of example ground-truth tracks and detections. Specifically, learning takes the form of bilevel optimization guided by our novel loss function. We evaluate our algorithm on simulated scenarios to understand its sensitivity to scenario aspects and model hyperparameters. Across varied scenario complexities, our method compares favorably to a strong baseline.

Problem

• Tracks multiple objects using kinematic and re-identification features
• Learns graph neural network parameters via bilevel optimization
• Evaluates tracking performance across varied scenario complexities

Innovation

• Graph neural network for edge cost computation
• Bilevel optimization for end-to-end learning
• Successive shortest paths algorithm for tracking