Encore: Conditioning Trajectory Forecasting via Biased Ego Rehearsals

2026-05-11

AI Summary

This work addresses the insufficient modeling of subjective factors—such as intent and behavioral preferences—in trajectory prediction by introducing a novel approach inspired by psychological counterfactual thinking. The method decouples prediction into two stages: “rehearsal” and “replay.” In the rehearsal stage, an egocentric predictor generates trajectories augmented with learnable bias terms that explicitly encode individual subjectivity. These biased trajectories then condition the replay stage to guide the final prediction, enabling controllable modeling of multi-agent interactions. To our knowledge, this is the first framework to formalize subjectivity as learnable bias trajectories and leverage them through a conditional generation mechanism to enhance both accuracy and interpretability. Experiments demonstrate state-of-the-art performance across multiple benchmark datasets and provide clear visualizations of individual behavioral preferences.

Abstract

Learning and representing the subjectivities of agents has become a challenging but crucial problem in the trajectory prediction task. Such subjectivities not only present specific spatial or temporal structures, but also are anisotropic for all interaction participants. Despite great efforts, it remains difficult to explicitly learn and forecast these subjectivities, let alone further modulate models' predictions through a specific ego's subjectivity. Inspired by prefactual thoughts in psychology and relevant theatrical concepts, we interpret such subjectivities in future trajectories as the continuous process from rehearsal to encore. In the rehearsal phase, the proposed ego predictor focuses on how each ego agent learns to derive and direct a set of explicitly biased rehearsal trajectories for all participants in the scene from the short-term observations. Then, these rehearsal trajectories serve as immediate controls to condition final predictions, providing direct yet distinct ego biases for the prediction network to simulate agents' various subjectivities. Experiments across datasets not only demonstrate a consistent improvement in the performance of the proposed \emph{Encore} trajectory prediction model but also provide clear interpretability regarding subjectivities as biased ego rehearsals.

Problem

• trajectory forecasting
• subjectivity
• ego bias
• agent interaction
• anisotropic behavior

Innovation

• trajectory forecasting
• subjectivity modeling
• biased ego rehearsals
• prefactual reasoning
• conditional prediction