HDiffTG: A Lightweight Hybrid Diffusion-Transformer-GCN Architecture for 3D Human Pose Estimation

2025-05-07

AI Summary

To address the low accuracy, poor robustness, and excessive model complexity of 3D human pose estimation under occlusion and noise, this paper proposes HDiffTG—a lightweight hybrid architecture. Methodologically, it introduces the first integration of denoising diffusion probabilistic models (DDPMs) with Transformer-GCN: the Transformer captures global spatiotemporal dependencies, the GCN encodes local skeletal topology, and the diffusion process enables skeletal-aware progressive denoising and refinement. We further design a parameter-efficient fine-tuning strategy and a redefined objective function. HDiffTG achieves state-of-the-art performance on MPI-INF-3DHP and delivers a balanced trade-off between accuracy and inference efficiency on Human3.6M. It demonstrates strong robustness to occlusion and sensor noise, while maintaining a compact model size and low latency. The code and pretrained models are publicly available.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Reasoning under Uncertainty: Relational Probabilistic Models

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

We propose HDiffTG, a novel 3D Human Pose Estimation (3DHPE) method that integrates Transformer, Graph Convolutional Network (GCN), and diffusion model into a unified framework. HDiffTG leverages the strengths of these techniques to significantly improve pose estimation accuracy and robustness while maintaining a lightweight design. The Transformer captures global spatiotemporal dependencies, the GCN models local skeletal structures, and the diffusion model provides step-by-step optimization for fine-tuning, achieving a complementary balance between global and local features. This integration enhances the model's ability to handle pose estimation under occlusions and in complex scenarios. Furthermore, we introduce lightweight optimizations to the integrated model and refine the objective function design to reduce computational overhead without compromising performance. Evaluation results on the Human3.6M and MPI-INF-3DHP datasets demonstrate that HDiffTG achieves state-of-the-art (SOTA) performance on the MPI-INF-3DHP dataset while excelling in both accuracy and computational efficiency. Additionally, the model exhibits exceptional robustness in noisy and occluded environments. Source codes and models are available at https://github.com/CirceJie/HDiffTG

Problem

• Improves 3D human pose estimation accuracy and robustness
• Integrates Transformer, GCN, and diffusion models for feature balance
• Reduces computational overhead while maintaining high performance

Innovation

• Integrates Transformer, GCN, and diffusion model
• Balances global and local features effectively
• Optimizes lightweight design without performance loss