Robust Self-Paced Hashing for Cross-Modal Retrieval with Noisy Labels

2025-01-03

AI Summary

To address hash performance degradation in large-scale cross-modal retrieval with noisy labels, this paper proposes a cognition-inspired Robust Self-Paced Hashing (RSPH) framework. RSPH is the first to integrate self-paced learning into cross-modal hashing, jointly employing Contrastive Hashing Learning (CHL) to mitigate semantic gaps and Center-Aggregation Learning (CAL) to suppress intra-class variance. It introduces a noise-tolerant self-paced mechanism that enables difficulty-driven dynamic sample selection and progressive hash optimization. Additionally, multi-modal consistency modeling and self-paced regularization are incorporated to further enhance robustness. Extensive experiments on multiple benchmark datasets demonstrate that RSPH significantly outperforms state-of-the-art methods, achieving up to an 8.2% improvement in mean Average Precision (mAP) under high-noise conditions—validating its strong robustness to label noise and superior generalization capability.

Technology Category

• Machine Learning: Multimodal Learning
• Computer Vision: Multi-modal Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Cross-modal hashing (CMH) has appeared as a popular technique for cross-modal retrieval due to its low storage cost and high computational efficiency in large-scale data. Most existing methods implicitly assume that multi-modal data is correctly labeled, which is expensive and even unattainable due to the inevitable imperfect annotations (i.e., noisy labels) in real-world scenarios. Inspired by human cognitive learning, a few methods introduce self-paced learning (SPL) to gradually train the model from easy to hard samples, which is often used to mitigate the effects of feature noise or outliers. It is a less-touched problem that how to utilize SPL to alleviate the misleading of noisy labels on the hash model. To tackle this problem, we propose a new cognitive cross-modal retrieval method called Robust Self-paced Hashing with Noisy Labels (RSHNL), which can mimic the human cognitive process to identify the noise while embracing robustness against noisy labels. Specifically, we first propose a contrastive hashing learning (CHL) scheme to improve multi-modal consistency, thereby reducing the inherent semantic gap. Afterward, we propose center aggregation learning (CAL) to mitigate the intra-class variations. Finally, we propose Noise-tolerance Self-paced Hashing (NSH) that dynamically estimates the learning difficulty for each instance and distinguishes noisy labels through the difficulty level. For all estimated clean pairs, we further adopt a self-paced regularizer to gradually learn hash codes from easy to hard. Extensive experiments demonstrate that the proposed RSHNL performs remarkably well over the state-of-the-art CMH methods.

Problem

• Cross-modal Hashing
• Label Noise
• Retrieval Accuracy

Innovation

• Robust Self-Paced Learning
• Hashing with Noisy Labels
• Cross-modal Hashing