HSTGFormer: Hyper Spatial-Temporal Graph Transformer for 3D Human Pose Estimation

2026-08-12

AI Summary

This work addresses the limitations of existing monocular 3D human pose estimation methods, which often decouple spatial and temporal modeling, thereby discarding frame-level structural information and neglecting the inherent spatiotemporal coupling in human motion. To overcome this, we propose HSTGFormer, a novel framework that formulates joint spatiotemporal reasoning as hypergraph aggregation over local joint–temporal nodes, constructing a Hyper Spatio-Temporal Graph (HSTG) to preserve structure-aware local motion features. Additionally, we introduce an Adaptive Dual-Scale Temporal Graph (ADSTG) to capture both short- and long-range dependencies specific to each joint, along with a lightweight node fusion module that adaptively integrates multi-graph representations. Extensive experiments on Human3.6M and MPI-INF-3DHP demonstrate that our approach achieves state-of-the-art accuracy and efficiency, surpassing conventional stage-wise modeling paradigms.

Abstract

Transformer-based methods have achieved strong performance in monocular 3D human pose estimation, but most existing approaches organise spatial and temporal reasoning as separate stages, which may weaken unified spatial-temporal interdependencies inherent in human motion and compress frame-level structural information before temporal modelling. In this paper, we propose HSTGFormer, a graph-enhanced Transformer framework that reformulates spatial-temporal reasoning as localised coupled graph aggregation over joint-time nodes. Specifically, HSTGFormer introduces a Hyper Spatial-Temporal Graph (HSTG), which decomposes global spatial-temporal reasoning into local spatial-temporal receptive fields around individual joint-time nodes by extending per-frame skeleton graphs into temporal neighbourhoods, thereby enabling structure-aware coupled reasoning while preserving local structural motion information. It further incorporates an Adaptive Dual-Scale Temporal Graph (ADSTG) to capture joint-specific temporal dependencies over complementary short- and long-range windows. A lightweight node-wise fusion module further adaptively integrates the two graph representations for each joint-time node. Experiments on Human3.6M and MPI-INF-3DHP show that HSTGFormer achieves strong accuracy with high computational efficiency.

Problem

• 3D human pose estimation
• spatial-temporal reasoning
• monocular vision
• graph representation
• structural motion information

Innovation

• Hyper Spatial-Temporal Graph
• Graph Transformer
• 3D Human Pose Estimation
• Adaptive Dual-Scale Temporal Graph
• Joint-Time Node