Rethinking Graph Convolution for 2D-to-3D Hand Pose Lifting

2026-05-13

AI Summary

This work addresses the inefficiency of hand topology modeling in 2D-to-3D hand pose lifting by introducing the hand skeletal structure as a soft prior within the spatial aggregation process, rather than relying on rigid adjacency constraints. Through controlled ablation studies, the authors systematically compare the modeling capabilities of graph convolution and self-attention mechanisms, demonstrating the superior performance of adaptive spatial attention. The proposed method integrates multi-head self-attention, graph attention networks, graph-distance-based positional encoding, and multi-hop adjacency graph convolution. Evaluated on the FPHA benchmark, it significantly reduces the mean per-joint position error (MPJPE) from 12.36 mm to 10.09 mm, validating the effectiveness of the proposed strategy in enhancing 3D hand pose estimation accuracy.

Abstract

Graph convolutional networks (GCNs) are widely used for 3D hand pose estimation, where the hand skeleton is encoded as a fixed adjacency graph. We revisit whether this is the most effective way to incorporate hand topology in 2D-to-3D lifting. In this paper, we perform controlled, parameter-matched ablations on the FPHA benchmark and show that standard multi-head self-attention consistently outperforms GCN baselines. Even when the GCN is strengthened with multi-hop adjacency and matched parameter count, self-attention reduces MPJPE from 12.36 mm to 10.09 mm. A skeleton-constrained graph attention network recovers most of this gap, indicating that input-dependent aggregation is a major source of improvement, while fully connected attention yields additional gains. We further show that hand topology is most effective when introduced as a soft structural prior through graph-distance positional encoding, rather than as a hard adjacency constraint. These results suggest that, for hand pose lifting, adaptive spatial attention is a more effective inductive bias than fixed graph convolution.

Problem

• hand pose lifting
• graph convolution
• hand topology
• 2D-to-3D pose estimation
• inductive bias

Innovation

• self-attention
• graph convolution
• hand pose lifting
• positional encoding
• adaptive spatial attention