MEgoVista: Multi-view Ego-aware Motion Estimation for Metric 4D Hands and Head in the Wild

2026-09-15

AI Summary

为了解决在自然环境中高精度手部动作重建的问题，MEgoVista通过单个未准备的头戴式视角记录，在重力对齐的世界坐标系中重建度量单位下的双手和头部运动。

Abstract

Learning manipulation from human video requires high-fidelity hand-motion reconstruction in metric units. Today's metric hand labels come from studio rigs and instrumented headsets, and both are confined in the same two ways: neither leaves a prepared setting, and neither is checked against an independent reference. Unconstrained head-worn recording promises the opposite trade-off, scaling with the number of people wearing a device. We therefore introduce MEgoVista, an offline pipeline that turns a single unprepared MEgo View recording into metric two-hand and head motion in one gravity-aligned world frame. Three properties set it apart from existing egocentric reconstruction systems: first, it reconstructs in settings studio volumes and tabletop rigs cannot reach, settling hand ownership at detection so bystander hands stay out of the wearer's trajectory; second, it takes its metric gauge from calibrated stereo rather than a monocular prior, installing scale at initialisation so policies receive physical units, not arbitrary coordinates; third, both outputs are scored inside a motion-capture volume against independent Chingmu optical capture, under a protocol that audits its own reference and charges what a method declines to predict. MEgoVista is offered as a measured route from egocentric video to metric hand supervision, one that widens where such labels can be gathered.

Problem

• hand-motion reconstruction
• metric units
• unconstrained head-worn recording

Innovation

• unconstrained head-worn recording
• calibrated stereo
• metric hand-motion reconstruction