EgoKit: Towards Unified Low-Cost Egocentric Data Collection with Heterogeneous Devices

2026-05-15

AI Summary

This work addresses the fragmentation in first-person video capture caused by heterogeneous devices and the lack of a unified, low-cost cross-platform solution. The authors propose an open-source toolkit supporting six device categories—including smartphones, tablets, smart glasses, and XR headsets—that enables consistent data collection through a cross-platform SDK, a standardized video interface, 26-joint hand tracking compliant with OpenXR, USB-C peripheral expansion, and a uniform logging format. For the first time, the system synchronously captures high-quality egocentric videos from both eye- and wrist-mounted perspectives across diverse hardware, while XR devices additionally provide aligned head pose and hand-tracking data. This approach substantially enhances compatibility and reproducibility in first-person data acquisition.

Abstract

Egocentric video is increasingly used as a data source for robot learning, activity understanding, and embodied AI research, but collecting it at scale remains fragmented in practice: each candidate host device, such as an Android phone, iPhone, iPad, smart glasses, or extended reality (XR) headset, exposes a different SDK, a different policy on raw camera access, and different limitations on external USB cameras and on-device tracking. Synchronized ego-view and wrist-view capture is therefore typically obtained by either committing to a single proprietary platform or building one-off rigs that do not transfer across devices. To address this gap, we present EgoKit, a toolkit that exposes the same egocentric recording workflow across six heterogeneous host devices. Across all supported devices, EgoKit presents the same recording interaction and produces locally stored video with a uniform log format; on XR headsets, it additionally logs head pose and OpenXR-standard 26-joint hand tracking aligned to the video streams. The companion accessories, including two wrist cameras with mounts, a head strap, and a USB-C hub, add wrist-view capture to any supported host without custom hardware fabrication. EgoKit is available at \url{https://egokit.chuange.org/}.

Problem

• egocentric video
• heterogeneous devices
• data collection
• synchronization
• device fragmentation

Innovation

• Egocentric data collection
• Cross-platform toolkit
• Heterogeneous devices
• Hand tracking
• Unified logging format