🤖 AI Summary
To address occlusion-induced missing pedestrian keypoints in autonomous driving, this paper proposes a disentanglement- and dimensionality-reduction-guided dual-GAN pose completion framework. Methodologically, it introduces a novel occlusion-region separation and feature dimensionality reduction preprocessing mechanism, coupled with a collaborative dual-generator architecture integrating Huber loss, residual connections, and L1 regularization for end-to-end high-accuracy, ultra-low-latency completion. Evaluated on MS COCO and JAAD datasets, the method reduces mean keypoint error by 18.7% over GAIN, k-NN, MissForest, and multiple interpolation baselines, with only 0.4 ms per frame—meeting stringent real-time requirements of automotive systems. This work is the first to jointly leverage dimensionality-reduction alignment and dual-GAN cooperative generation for occluded keypoint completion, achieving an unprecedented balance among accuracy, robustness, and deployment efficiency.
📝 Abstract
To mitigate the challenges arising from partial occlusion in human pose keypoint based pedestrian detection methods , we present a novel pedestrian pose keypoint completion method called the separation and dimensionality reduction-based generative adversarial imputation networks (SDR-GAIN) . Firstly, we utilize OpenPose to estimate pedestrian poses in images. Then, we isolate the head and torso keypoints of pedestrians with incomplete keypoints due to occlusion or other factors and perform dimensionality reduction to enhance features and further unify feature distribution. Finally, we introduce two generative models based on the generative adversarial networks (GAN) framework, which incorporate Huber loss, residual structure, and L1 regularization to generate missing parts of the incomplete head and torso pose keypoints of partially occluded pedestrians, resulting in pose completion. Our experiments on MS COCO and JAAD datasets demonstrate that SDR-GAIN outperforms basic GAIN framework, interpolation methods PCHIP and MAkima, machine learning methods k-NN and MissForest in terms of pose completion task. Furthermore, the SDR-GAIN algorithm exhibits a remarkably short running time of approximately 0.4ms and boasts exceptional real-time performance. As such, it holds significant practical value in the domain of autonomous driving, wherein high system response speeds are of paramount importance. Specifically, it excels at rapidly and precisely capturing human pose key points, thus enabling an expanded range of applications for pedestrian detection tasks based on pose key points, including but not limited to pedestrian behavior recognition and prediction.
