🤖 AI Summary
Transformer-based approaches for multi-agent trajectory prediction in autonomous driving exhibit strong modeling capacity but suffer from training instability and high computational overhead due to layer normalization.
Method: This paper proposes a normalization-free Transformer architecture: (i) replacing layer normalization with a dynamic Tanh activation—introduced to trajectory prediction for the first time—and (ii) designing a lightweight snapshot ensemble framework that synergistically integrates cyclical learning rate scheduling with model weight averaging.
Contribution/Results: Evaluated on the Argoverse dataset, our method achieves significant improvements: 8.2% reduction in average displacement error (ADE) and 6.7% reduction in final displacement error (FDE), 23% faster inference speed, and enhanced robustness in complex traffic scenarios. The proposed approach establishes a new paradigm for efficient, stable, and deployable trajectory prediction—eliminating normalization-induced bottlenecks while maintaining high accuracy and generalization.
📝 Abstract
Accurate trajectory prediction is a cornerstone for the safe operation of autonomous driving systems, where understanding the dynamic behavior of surrounding agents is crucial. Transformer-based architectures have demonstrated significant promise in capturing complex spatio-temporality dependencies. However, their reliance on normalization layers can lead to computation overhead and training instabilities. In this work, we present a two-fold approach to address these challenges. First, we integrate DynamicTanh (DyT), which is the latest method to promote transformers, into the backbone, replacing traditional layer normalization. This modification simplifies the network architecture and improves the stability of the inference. We are the first work to deploy the DyT to the trajectory prediction task. Complementing this, we employ a snapshot ensemble strategy to further boost trajectory prediction performance. Using cyclical learning rate scheduling, multiple model snapshots are captured during a single training run. These snapshots are then aggregated via simple averaging at inference time, allowing the model to benefit from diverse hypotheses without incurring substantial additional computational cost. Extensive experiments on Argoverse datasets demonstrate that our combined approach significantly improves prediction accuracy, inference speed and robustness in diverse driving scenarios. This work underscores the potential of normalization-free transformer designs augmented with lightweight ensemble techniques in advancing trajectory forecasting for autonomous vehicles.