🤖 AI Summary
RGB-infrared cross-modal person re-identification suffers from severe domain shift due to modality discrepancy and non-overlapping identity classes between training and testing sets, leading to overfitting on seen classes and poor generalization to unseen ones. To address this, we propose the Multi-Granularity Memory-Regulated Alignment (MG-MRA) module, which jointly models latent variables from fine-grained to coarse semantic levels, employs sparse global structural hashing for similarity measurement, and enforces cross-modal feature alignment coupled with adversarial domain-distribution regularization to mitigate feature-level shifts. Evaluated on RegDB and SYSU-MM01, MG-MRA achieves significant improvements over state-of-the-art methods, demonstrating superior zero-shot cross-domain generalization and robustness. Its core innovation lies in the synergistic integration of multi-granularity latent representation learning, memory-augmented regulation, and structured hashing—collectively enhancing transferability under unseen-class, cross-modal, and cross-domain settings.
📝 Abstract
The domain shift, coming from unneglectable modality gap and non-overlapped identity classes between training and test sets, is a major issue of RGB-Infrared person re-identification. A key to tackle the inherent issue -- domain shift -- is to enforce the data distributions of the two domains to be similar. However, RGB-IR ReID always demands discriminative features, leading to over-rely feature sensitivity of seen classes, extit{e.g.}, via attention-based feature alignment or metric learning. Therefore, predicting the unseen query category from predefined training classes may not be accurate and leads to a sub-optimal adversarial gradient. In this paper, we uncover it in a more explainable way and propose a novel multi-granularity memory regulation and alignment module (MG-MRA) to solve this issue. By explicitly incorporating a latent variable attribute, from fine-grained to coarse semantic granularity, into intermediate features, our method could alleviate the over-confidence of the model about discriminative features of seen classes. Moreover, instead of matching discriminative features by traversing nearest neighbor, sparse attributes, extit{i.e.}, global structural pattern, are recollected with respect to features and assigned to measure pair-wise image similarity in hashing. Extensive experiments on RegDB cite{RegDB} and SYSU-MM01 cite{SYSU} show the superiority of the proposed method that outperforms existing state-of-the-art methods. Our code is available in https://github.com/Chenfeng1271/MGMRA.
