Hand-4DGS: Feed-Forward 3D Gaussian Splatting for 4D Hand Reconstruction from Egocentric Videos

2026-06-17

AI Summary

This work addresses the challenges of 4D hand reconstruction from first-person videos—such as rapid head motion, dynamic hand articulation, severe occlusions, and monocular ambiguities—by proposing the first feed-forward framework capable of directly reconstructing high-fidelity 4D hand models without requiring ground-truth 3D hand poses. The method integrates a mesh-guided 3D Gaussian splatting representation with a temporal convolutional network, enabling efficient training under pure 2D image supervision. Evaluated on the H2O and ARCTIC datasets, the framework significantly outperforms existing approaches while achieving real-time inference at approximately 60 FPS and demonstrating strong generalization capability. To the best of our knowledge, this is the first method to achieve high-quality, feed-forward 4D hand reconstruction without any 3D annotations.

Abstract

Dynamic 3D hand reconstruction from egocentric videos is essential for next-generation computing platforms such as AR/VR and AI glasses. Despite its importance, most prior works focus either on multi-view 3D hand reconstruction or on 4D human body reconstruction. Egocentric 4D hand reconstruction remains challenging due to fast head motion, rapid hand dynamics, severe occlusions, and inherent ambiguity from single-view observations. To address these challenges, we introduce Hand-4DGS, the first feed-forward framework for reconstructing dynamic 4D hands directly from egocentric videos, enabling both fast (~60 FPS) inference and strong generalization. Our approach incorporates a mesh-guided representation for structural priors and temporal convolutions to model dynamic motion. We evaluate our framework on two challenging egocentric datasets, H2O and ARCTIC, and demonstrate significant improvements over baselines. Our method benefits from the generalization capability of feed-forward networks and effective 2D image supervision through Gaussian splatting, without requiring expensive 3D hand pose ground-truth annotations.

Problem

• 4D hand reconstruction
• egocentric videos
• dynamic hand modeling
• single-view ambiguity
• occlusion

Innovation

• 4D hand reconstruction
• egocentric video
• 3D Gaussian Splatting
• feed-forward network
• mesh-guided representation