Learning Through Retrospection: Improving Trajectory Prediction for Automated Driving with Error Feedback

2025-04-18

AI Summary

Current trajectory prediction models treat each frame’s prediction as an independent instantaneous task, lacking error correction during inference—leading to error accumulation and poor long-term consistency. To address this, we propose an inference-time closed-loop backtracking learning mechanism: the first dynamic reflection-and-correction paradigm for trajectory prediction that leverages historical error feedback. Our approach employs a temporal state reflection module to explicitly model prediction deviations, integrates error-aggregated feedback, and adopts closed-loop rollout training to enable autonomous identification and correction of systematic errors during inference. The method is jointly optimized on nuScenes and Argoverse benchmarks, achieving a 31.9% reduction in minimum average displacement error (minADE). It significantly enhances robustness in long-horizon prediction and generalization to out-of-distribution scenarios—such as those involving undetected traffic participants—demonstrating improved real-world deployability.

Technology Category

• Humans and AI: Human-Aware Planning and Behavior Prediction
• Search and Optimization: Learning to Search
• Intelligent Robots: State Estimation

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

In automated driving, predicting trajectories of surrounding vehicles supports reasoning about scene dynamics and enables safe planning for the ego vehicle. However, existing models handle predictions as an instantaneous task of forecasting future trajectories based on observed information. As time proceeds, the next prediction is made independently of the previous one, which means that the model cannot correct its errors during inference and will repeat them. To alleviate this problem and better leverage temporal data, we propose a novel retrospection technique. Through training on closed-loop rollouts the model learns to use aggregated feedback. Given new observations it reflects on previous predictions and analyzes its errors to improve the quality of subsequent predictions. Thus, the model can learn to correct systematic errors during inference. Comprehensive experiments on nuScenes and Argoverse demonstrate a considerable decrease in minimum Average Displacement Error of up to 31.9% compared to the state-of-the-art baseline without retrospection. We further showcase the robustness of our technique by demonstrating a better handling of out-of-distribution scenarios with undetected road-users.

Problem

• Improving trajectory prediction in automated driving using error feedback
• Enabling models to correct systematic errors during inference
• Enhancing robustness in out-of-distribution scenarios for road-users

Innovation

• Uses retrospection to improve trajectory predictions
• Learns from error feedback during inference
• Reduces displacement error by 31.9%