Annealed Winner-Takes-All for Motion Forecasting

2024-09-17

arXiv.org

AI Summary

Existing multi-hypothesis methods for autonomous driving multi-agent motion prediction suffer from initialization sensitivity, training instability, and high post-processing overhead during inference. To address these issues, this paper proposes the annealed Winner-Takes-All (aWTA) loss function—the first to incorporate annealed Monte Carlo Localization (MCL) into trajectory prediction. By leveraging an end-to-end differentiable annealing mechanism, aWTA optimizes a small set of candidate trajectories, jointly enhancing both predictive diversity and accuracy without requiring large-scale hypothesis generation or post-hoc selection. The method is agnostic to backbone architectures and integrates seamlessly with mainstream models such as Transformers and CNN-LSTMs. We release open-source, plug-and-play code. On nuScenes and Argoverse benchmarks, our approach achieves significant reductions in minFDE and minADE while improving inference efficiency by over 30%.

Technology Category

• Multiagent Systems: Adversarial Agents
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Machine Learning: Hardware-aware ML

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

In autonomous driving, motion prediction aims at forecasting the future trajectories of nearby agents, helping the ego vehicle to anticipate behaviors and drive safely. A key challenge is generating a diverse set of future predictions, commonly addressed using data-driven models with Multiple Choice Learning (MCL) architectures and Winner-Takes-All (WTA) training objectives. However, these methods face initialization sensitivity and training instabilities. Additionally, to compensate for limited performance, some approaches rely on training with a large set of hypotheses, requiring a post-selection step during inference to significantly reduce the number of predictions. To tackle these issues, we take inspiration from annealed MCL, a recently introduced technique that improves the convergence properties of MCL methods through an annealed Winner-Takes-All loss (aWTA). In this paper, we demonstrate how the aWTA loss can be integrated with state-of-the-art motion forecasting models to enhance their performance using only a minimal set of hypotheses, eliminating the need for the cumbersome post-selection step. Our approach can be easily incorporated into any trajectory prediction model normally trained using WTA and yields significant improvements. To facilitate the application of our approach to future motion forecasting models, the code is made publicly available: https://github.com/valeoai/MF_aWTA.

Problem

• Addresses initialization sensitivity in motion forecasting models.
• Reduces training instabilities in trajectory prediction systems.
• Eliminates need for post-selection step in hypothesis reduction.

Innovation

• Annealed Winner-Takes-All loss improves convergence
• Minimizes hypotheses without post-selection step
• Compatible with existing trajectory prediction models