mmGAT: Pose Estimation by Graph Attention with Mutual Features from mmWave Radar Point Cloud

2024-06-09

ICC 2024 - IEEE International Conference on Communications

AI Summary

This work addresses the performance limitations of image-based human pose estimation in low-light, dark environments, and privacy-sensitive scenarios by proposing a robust and privacy-preserving solution based on millimeter-wave radar. By integrating Graph Attention Networks (GATs) with a cross-correlation feature extraction mechanism, the method achieves high-precision human pose modeling directly on radar point clouds for the first time. It effectively captures fine-grained spatial relationships inherent in human body structure. Evaluated on two public radar datasets, the approach establishes a new state-of-the-art, reducing Mean Per-Joint Position Error (MPJPE) by 35.6% and Protocol-Aware MPJPE (PA-MPJPE) by 14.1%, thereby significantly overcoming the environmental and privacy constraints that hinder conventional vision-based methods.

Technology Category

• Computer Vision: Biometrics, Face, Gesture & Pose
• Machine Learning: Privacy
• Intelligent Robots: State Estimation

Application Category

• Security and Privacy: Large-scale security measurements
• User Modeling, Personalization and Recommendation: User privacy protection in personalized systems
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

Pose estimation and human action recognition (HAR) are pivotal technologies spanning various domains. While the image-based pose estimation and HAR are widely admired for their superior performance, they lack in privacy protection and suboptimal performance in low-light and dark environments. This paper exploits the capabilities of millimeter-wave (mmWave) radar technology for human pose estimation by processing radar data with Graph Neural Network (GNN) architecture, coupled with the attention mechanism. Our goal is to capture the finer details of the radar point cloud to improve the pose estimation performance. To this end, we present a unique feature extraction technique that exploits the full potential of the GNN processing method for pose estimation. Our model mmGAT demonstrates remarkable performance on two publicly available benchmark mmWave datasets and establishes new state of the art results in most scenarios in terms of human pose estimation. Our approach achieves a noteworthy reduction of pose estimation mean per joint position error (MPJPE) by 35.6% and PA-MPJPE by 14.1% from the current state of the art benchmark within this domain.

Problem

• pose estimation
• mmWave radar
• privacy protection
• low-light environment
• human action recognition

Innovation

• mmWave radar
• Graph Attention Network
• pose estimation
• mutual features
• point cloud