GEA: Generation-Enhanced Alignment for Text-to-Image Person Retrieval

2025-11-13

AI Summary

To address incomplete semantic descriptions, large modality gaps, and few-shot overfitting in text-to-image person retrieval (TIPR), this paper proposes a Generative Enhancement Alignment (GEA) framework. GEA leverages high-fidelity images synthesized by diffusion models as intermediate representations, incorporating a text-guided token enhancement module and a text–generated image–real image ternary cross-attention fusion module. A novel ternary alignment loss is introduced to enforce unified cross-modal representation learning. Unlike methods relying solely on raw images or shallow feature mappings, GEA effectively bridges semantic and modality discrepancies while enhancing fine-grained cross-modal alignment. Extensive experiments demonstrate state-of-the-art performance on three benchmark datasets—CUHK-PEDES, RSTPReid, and ICFG-PEDES—validating both the effectiveness and strong generalizability of the proposed approach.

Technology Category

• Computer Vision: Multi-modal Vision
• Natural Language Processing: Language Grounding & Multi-modal NLP
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Text-to-Image Person Retrieval (TIPR) aims to retrieve person images based on natural language descriptions. Although many TIPR methods have achieved promising results, sometimes textual queries cannot accurately and comprehensively reflect the content of the image, leading to poor cross-modal alignment and overfitting to limited datasets. Moreover, the inherent modality gap between text and image further amplifies these issues, making accurate cross-modal retrieval even more challenging. To address these limitations, we propose the Generation-Enhanced Alignment (GEA) from a generative perspective. GEA contains two parallel modules: (1) Text-Guided Token Enhancement (TGTE), which introduces diffusion-generated images as intermediate semantic representations to bridge the gap between text and visual patterns. These generated images enrich the semantic representation of text and facilitate cross-modal alignment. (2) Generative Intermediate Fusion (GIF), which combines cross-attention between generated images, original images, and text features to generate a unified representation optimized by triplet alignment loss. We conduct extensive experiments on three public TIPR datasets, CUHK-PEDES, RSTPReid, and ICFG-PEDES, to evaluate the performance of GEA. The results justify the effectiveness of our method. More implementation details and extended results are available at https://github.com/sugelamyd123/Sup-for-GEA.

Problem

• Addresses inaccurate text queries in person image retrieval
• Bridges modality gap between text descriptions and images
• Reduces overfitting to limited datasets in cross-modal alignment

Innovation

• Introduces diffusion-generated images as semantic bridges
• Combines cross-attention across generated and original images
• Uses triplet alignment loss for unified representation optimization