Unsupervised Data-Efficient Cross-Modal Retrieval with Global-Neighborhood Alignment Hashing

2026-06-30

AI Summary

This work addresses the high data acquisition cost of existing unsupervised cross-modal hashing methods, which typically rely on large-scale image-text pairs. To overcome this limitation, we propose Global-Neighborhood Aligned Hashing (GNAH), a novel approach that effectively transfers the semantic structure of vision-language foundation models into a compact binary Hamming space under limited paired data. GNAH integrates a prototype-anchored global alignment module with a contrastive stochastic neighborhood alignment module, jointly preserving global semantic consistency and local structural relationships to mitigate overfitting in sparse pairing scenarios. Extensive experiments demonstrate that GNAH significantly outperforms state-of-the-art unsupervised cross-modal retrieval methods under data-constrained settings, highlighting its practical utility.

Abstract

Compared to supervised cross-modal hashing (CMH), unsupervised CMH reduces the reliance on manual labeling by learning binary codes from unlabeled image-text pairs. However, existing unsupervised CMH methods often rely on large-scale image-text pairs, which are costly to collect. To address this limitation, we propose Global-Neighborhood Alignment Hashing (GNAH), a novel approach that preserves the semantic structure of vision-language foundation models within a compact binary Hamming space using only a limited number of image-text pairs. Specifically, GNAH captures global structural information from the continuous latent space and transfers it into the binary Hamming space through a Prototype-Anchored Global Alignment module. In addition, GNAH extends conventional pairwise contrastive learning by modeling stochastic neighborhood relationships via a Contrastive Stochastic Neighborhood Alignment module, thereby alleviating overfitting to sparse pairwise correlations. Extensive experiments demonstrate that GNAH consistently outperforms existing unsupervised cross-modal retrieval methods under data-constrained settings, offering a practical solution for real-world CMH applications.

Problem

• unsupervised cross-modal hashing
• data efficiency
• image-text retrieval
• limited data

Innovation

• unsupervised cross-modal hashing
• data-efficient learning
• global-neighborhood alignment
• binary Hamming space
• contrastive stochastic neighborhood