EgoTraj: Real-World Egocentric Human Trajectory Dataset for Multimodal Prediction

2026-05-18

AI Summary

This work addresses the lack of multimodal egocentric human trajectory datasets captured in real-world urban environments, which limits the accuracy of behavior prediction. To bridge this gap, we introduce EgoTraj, an open dataset comprising 75 user-guided navigation sequences collected in diverse city settings using Meta Quest Pro headsets. The dataset synchronously records RGB video, six-degree-of-freedom head pose, 3D gaze vectors, and semantic scene annotations, representing the first effort to capture long-duration, self-directed, multimodal trajectories from multiple users across varied urban routes. We publicly release the dataset, accompanying code, and a visualization tool—EgoViz Dashboard—and validate through benchmark experiments that gaze, scene context, and motion cues significantly enhance trajectory prediction performance, thereby establishing a new foundation for research in AR perception and assistive navigation.

Abstract

Accurately forecasting human trajectories from an egocentric perspective plays a central role in applications such as humanoid robotics, wearable sensing systems, and assistive navigation. However, progress in this direction remains limited due to the scarcity of egocentric trajectory datasets collected in real-world environments. Addressing this need, we introduce EgoTraj, an egocentric multimodal open dataset recorded using Meta Quest Pro (MQPro). EgoTraj contains 75 sequences of human navigation collected from multiple MQPro wearers in real-world urban environments. Each recording provides synchronized RGB video along with ground-truth data, including continuous time-synchronized 6-degree-of-freedom head poses, per-frame 3D eye gaze vectors, scene annotations. To the best of our knowledge, EgoTraj differs from typical egocentric trajectory datasets by capturing long-horizon, self-directed navigation across diverse urban routes with broad participant diversity. To demonstrate the potential of the dataset, we benchmark several state-of-the-art methods for egocentric trajectory prediction and conduct ablation studies to analyze the contributions of gaze, scene, and motion cues. The results highlight the utility of EgoTraj for AR-based perception, navigation, and assistive systems. The EgoTraj dataset, code, and EgoViz Dashboard are publicly available at https://github.com/yehiahmad/EgoTraj.

Problem

• egocentric trajectory prediction
• real-world dataset
• human navigation
• multimodal perception
• AR-based systems

Innovation

• egocentric trajectory prediction
• multimodal dataset
• real-world navigation
• 3D gaze estimation
• AR-based assistive systems