L2A: Learning to Accumulate Pose History for Accurate 3D Human Pose Estimation

2026-05-09

AI Summary

This work addresses the limitations of existing 2D-to-3D human pose estimation methods, which struggle to effectively leverage historical pose information from deeper network layers and are constrained by fixed residual connections that hinder the reuse of fine-grained spatial structures and short-term motion cues from early layers. To overcome these issues, the authors propose a history-aware framework built upon a spatiotemporal parallel Transformer backbone to maintain representation consistency. The approach introduces two novel components: a History Pose Accumulation (HPA) mechanism and a Layer-wise Pose History Aggregation (LPA) module, enabling adaptive fusion of cross-layer features. This is the first method to explicitly emphasize the importance of cross-layer historical pose representation consistency, achieving state-of-the-art performance on multiple benchmark datasets and significantly improving 3D human pose estimation accuracy.

Abstract

Existing 2D-3D lifting human pose estimation methods have achieved strong performance. But the utilization of historical pose representations across network depth was overlooked. In current pipelines, information is propagated through fixed residual connections, which restricts effective reuse of early-layer features such as fine-grained spatial structures and short-term motion cues. However, naively incorporating historical features across layers is non-trivial. We further identify that maintaining a consistent representation space across layers is a prerequisite for effective cross-layer feature aggregation. To address this issue, we propose a history-aware framework that enables effective network cross-layer history feature utilization. Specifically, we adopt a spatial-temporal parallel Transformer backbone to prevent alternating spatial-temporal transformations during sequential processing, thereby maintaining a consistent representation space. Building upon this, we introduce a History Pose Accumulation (HPA) mechanism that adaptively aggregates features from all preceding layers to enhance current representations. Furthermore, we propose a Layer Pose History Aggregation (LPA) module that transforms layer pose features into a compact and structured form, reducing redundancy and enabling more stable aggregation. Extensive experiments demonstrate that our approach achieves state-of-the-art performance on benchmarks.

Problem

• 3D human pose estimation
• pose history
• cross-layer feature aggregation
• representation consistency
• 2D-3D lifting

Innovation

• History Pose Accumulation
• Layer Pose History Aggregation
• Spatial-Temporal Parallel Transformer
• Cross-layer Feature Aggregation
• 3D Human Pose Estimation