NanoHTNet: Nano Human Topology Network for Efficient 3D Human Pose Estimation

2025-01-27

AI Summary

3D human pose estimation on edge devices (e.g., Jetson Nano) faces a fundamental trade-off among accuracy, latency, and resource efficiency. To address this, we propose Hierarchical Mixer, the first architecture to explicitly model the physical skeletal topology and implicitly capture motion priors via hierarchical spatial-temporal mixers. We further introduce Efficient Temporal-Spatial Tokenization (ETST) and PoseCLR—a contrastive learning framework leveraging both explicit and implicit structural priors—alongside DCT-based low-pass filtering to enhance temporal robustness. The resulting model achieves significant reductions in parameter count (−47%) and computational cost while enabling real-time inference (<30 ms) on Jetson Nano. On Human3.6M and other benchmarks, it outperforms state-of-the-art methods in accuracy (MPJPE reduced by 3.2%), latency, and model compactness, establishing a new paradigm for efficient, high-fidelity 3D pose estimation at the edge.

Technology Category

• Computer Vision: 3D Computer Vision
• Machine Learning: Learning on the Edge & Model Compression
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

The widespread application of 3D human pose estimation (HPE) is limited by resource-constrained edge devices, requiring more efficient models. A key approach to enhancing efficiency involves designing networks based on the structural characteristics of input data. However, effectively utilizing the structural priors in human skeletal inputs remains challenging. To address this, we leverage both explicit and implicit spatio-temporal priors of the human body through innovative model design and a pre-training proxy task. First, we propose a Nano Human Topology Network (NanoHTNet), a tiny 3D HPE network with stacked Hierarchical Mixers to capture explicit features. Specifically, the spatial Hierarchical Mixer efficiently learns the human physical topology across multiple semantic levels, while the temporal Hierarchical Mixer with discrete cosine transform and low-pass filtering captures local instantaneous movements and global action coherence. Moreover, Efficient Temporal-Spatial Tokenization (ETST) is introduced to enhance spatio-temporal interaction and reduce computational complexity significantly. Second, PoseCLR is proposed as a general pre-training method based on contrastive learning for 3D HPE, aimed at extracting implicit representations of human topology. By aligning 2D poses from diverse viewpoints in the proxy task, PoseCLR aids 3D HPE encoders like NanoHTNet in more effectively capturing the high-dimensional features of the human body, leading to further performance improvements. Extensive experiments verify that NanoHTNet with PoseCLR outperforms other state-of-the-art methods in efficiency, making it ideal for deployment on edge devices like the Jetson Nano. Code and models are available at https://github.com/vefalun/NanoHTNet.

Problem

• 3D Human Pose Estimation
• Edge Devices
• Efficient Model

Innovation

• NanoHTNet
• PoseCLR
• 3D Human Pose Estimation