CAMeL: Cross-modality Adaptive Meta-Learning for Text-based Person Retrieval

2025-04-26

AI Summary

In text-to-image person retrieval, synthetic data induces cross-modal domain shift between textual and visual modalities, severely degrading the generalization of pre-trained models and hindering downstream transfer. To address this, we propose a cross-modal adaptive meta-learning framework. Our key contributions are: (1) a dynamic erroneous sample memory module that explicitly models task-wise discriminative biases across modalities; and (2) an adaptive dual-rate parameter update mechanism that balances multi-task knowledge transfer with stability over historical tasks. Crucially, the framework requires no real annotations—only synthetic data—enabling robust pre-training. Extensive experiments demonstrate state-of-the-art performance on three benchmarks: CUHK-PEDES, ICFG-PEDES, and RSTPReid. Moreover, our method exhibits strong robustness to image-domain bias and noisy text annotations, alongside superior scalability to diverse synthetic data distributions.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Multimodal Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

Text-based person retrieval aims to identify specific individuals within an image database using textual descriptions. Due to the high cost of annotation and privacy protection, researchers resort to synthesized data for the paradigm of pretraining and fine-tuning. However, these generated data often exhibit domain biases in both images and textual annotations, which largely compromise the scalability of the pre-trained model. Therefore, we introduce a domain-agnostic pretraining framework based on Cross-modality Adaptive Meta-Learning (CAMeL) to enhance the model generalization capability during pretraining to facilitate the subsequent downstream tasks. In particular, we develop a series of tasks that reflect the diversity and complexity of real-world scenarios, and introduce a dynamic error sample memory unit to memorize the history for errors encountered within multiple tasks. To further ensure multi-task adaptation, we also adopt an adaptive dual-speed update strategy, balancing fast adaptation to new tasks and slow weight updates for historical tasks. Albeit simple, our proposed model not only surpasses existing state-of-the-art methods on real-world benchmarks, including CUHK-PEDES, ICFG-PEDES, and RSTPReid, but also showcases robustness and scalability in handling biased synthetic images and noisy text annotations. Our code is available at https://github.com/Jahawn-Wen/CAMeL-reID.

Problem

• Addressing domain biases in synthesized data for text-based person retrieval
• Enhancing model generalization via cross-modality adaptive meta-learning
• Improving robustness to biased synthetic images and noisy text annotations

Innovation

• Cross-modality Adaptive Meta-Learning framework
• Dynamic error sample memory unit
• Adaptive dual-speed update strategy