Adept: Annotation-denoising auxiliary tasks with discrete cosine transform map and keypoint for human-centric pretraining

2024-10-01

Neurocomputing

AI Summary

Pretraining以人为中心 models on RGB images suffers from poor data scalability and severe annotation noise due to the absence of depth information. Method: This paper proposes a depth-agnostic frequency-domain semantic learning framework. Its core innovation is a novel dual-path annotation purification mechanism that jointly leverages DCT feature maps and keypoint constraints, integrated with keypoint-guided contrastive learning, multi-scale denoising auxiliary tasks, and self-supervised pose consistency regularization—enabling robust representation learning under weak supervision. Contribution/Results: The method achieves significant performance gains on downstream tasks across benchmarks including PoseTrack and LaRa. Notably, even under 30% label noise, the pretrained model retains over 92% of its original accuracy, demonstrating exceptional robustness and generalization capability.

Technology Category

• Computer Vision: Motion & Tracking
• Machine Learning: Multimodal Learning
• Search and Optimization: Learning to Search

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Large pretrained models with web data

Problem

• Improving human-centric pretraining without depth data
• Enhancing RGB image semantic analysis via DCT
• Reducing annotation noise with keypoints and DCT maps

Innovation

• Uses Discrete Cosine Transform for semantic analysis
• Proposes annotation-denoising auxiliary tasks
• Leverages keypoints and DCT maps for fine-grained learning