🤖 AI Summary
To address two critical challenges in text-to-image person retrieval—cross-modal misalignment (caused by annotation errors and poor image quality) and semantic inconsistency induced by random masking—this paper proposes a noise-robust retrieval framework. Methodologically, it introduces three key components: (1) a Bidirectional Similarity Distribution Matching (BSDM) loss that explicitly models and suppresses interference from misaligned samples; (2) a Weight-Adaptive Focal (WAF) loss to enhance discriminative learning for hard samples; and (3) an EMA-guided attention-weighted selective masking mechanism that dynamically selects salient image regions and textual tokens while preserving core semantics. Extensive experiments on benchmark datasets—including CUHK-PEDES and RSTPReid—demonstrate significant improvements in retrieval accuracy, with mAP gains of 3.2–5.8% over state-of-the-art methods. The proposed framework further exhibits superior robustness to label noise and stronger generalization capability.
📝 Abstract
Most existing text-to-image person retrieval methods usually assume that the training image-text pairs are perfectly aligned; however, the noisy correspondence(NC) issue (i.e., incorrect or unreliable alignment) exists due to poor image quality and labeling errors. Additionally, random masking augmentation may inadvertently discard critical semantic content, introducing noisy matches between images and text descriptions. To address the above two challenges, we propose a noise label suppression method to mitigate NC and an Attention-Weighted Selective Mask (AWM) strategy to resolve the issues caused by random masking. Specifically, the Bidirectional Similarity Distribution Matching (BSDM) loss enables the model to effectively learn from positive pairs while preventing it from over-relying on them, thereby mitigating the risk of overfitting to noisy labels. In conjunction with this, Weight Adjustment Focal (WAF) loss improves the model's ability to handle hard samples. Furthermore, AWM processes raw images through an EMA version of the image encoder, selectively retaining tokens with strong semantic connections to the text, enabling better feature extraction. Extensive experiments demonstrate the effectiveness of our approach in addressing noise-related issues and improving retrieval performance.