Ego-1K -- A Large-Scale Multiview Video Dataset for Egocentric Vision

2026-03-14

AI Summary

This work addresses the scarcity of large-scale, temporally synchronized multi-view data for first-person dynamic scene reconstruction and novel-view synthesis. To this end, we introduce Ego-1K, a dataset comprising nearly 1,000 high-synchronization first-person multi-view video sequences captured using a custom VR headset equipped with 12 surrounding cameras and 4 head-mounted cameras, focusing on close-range dynamic interactions such as hand motions and hand-object manipulations. Ego-1K provides the first large-scale, high-quality benchmark tailored to complex egocentric interactions, ensured by precise spatiotemporal calibration and an automated processing pipeline for data alignment and usability. Experiments demonstrate that Ego-1K poses significant challenges to existing 3D/4D novel-view synthesis methods, effectively exposing their limitations under large viewpoint disparities and rapid motion, thereby establishing a critical foundation for future research.

Technology Category

• Computer Vision: Motion & Tracking
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Machine Learning: Multi-instance/Multi-view Learning

Application Category

• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Security and Privacy: Large-scale security measurements
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

We present Ego-1K, a large-scale collection of time-synchronized egocentric multiview videos designed to advance neural 3D video synthesis and dynamic scene understanding. The dataset contains nearly 1,000 short egocentric videos captured with a custom rig with 12 synchronized cameras surrounding a 4-camera VR headset worn by the user. Scene content focuses on hand motions and hand-object interactions in different settings. We describe rig design, data processing, and calibration. Our dataset enables new ways to benchmark egocentric scene reconstruction methods, an important research area as smart glasses with multiple cameras become omnipresent. Our experiments demonstrate that our dataset presents unique challenges for existing 3D and 4D novel view synthesis methods due to large disparities and image motion caused by close dynamic objects and rig egomotion. Our dataset supports future research in this challenging domain. It is available at https://huggingface.co/datasets/facebook/ego-1k.

Problem

• egocentric vision
• multiview video
• 3D video synthesis
• dynamic scene understanding
• novel view synthesis

Innovation

• egocentric vision
• multiview video dataset
• neural 3D synthesis
• hand-object interaction
• novel view synthesis