CLID-MU: Cross-Layer Information Divergence Based Meta Update Strategy for Learning with Noisy Labels

2025-07-15

AI Summary

Meta-learning struggles with label noise when clean annotated meta-datasets are unavailable. Method: This paper proposes a clean-label-free meta-learning framework that leverages structural consistency discrepancies between the last hidden layer and output layer to quantify cross-layer information divergence for noise sample identification; it further introduces a meta-update mechanism grounded in structural alignment and consistency constraints to achieve noise-robust optimization. Contribution/Results: The method requires neither clean labels nor auxiliary metadata. Evaluated on CIFAR-10, CIFAR-100, and WebVision under diverse synthetic and real-world label noise settings, it consistently outperforms state-of-the-art approaches, demonstrating superior effectiveness, generalization, and robustness.

Technology Category

• Machine Learning: Adversarial Learning & Robustness
• Computer Vision: Adversarial Attacks & Robustness
• Search and Optimization: Learning to Search

Application Category

• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs

Abstract

Learning with noisy labels (LNL) is essential for training deep neural networks with imperfect data. Meta-learning approaches have achieved success by using a clean unbiased labeled set to train a robust model. However, this approach heavily depends on the availability of a clean labeled meta-dataset, which is difficult to obtain in practice. In this work, we thus tackle the challenge of meta-learning for noisy label scenarios without relying on a clean labeled dataset. Our approach leverages the data itself while bypassing the need for labels. Building on the insight that clean samples effectively preserve the consistency of related data structures across the last hidden and the final layer, whereas noisy samples disrupt this consistency, we design the Cross-layer Information Divergence-based Meta Update Strategy (CLID-MU). CLID-MU leverages the alignment of data structures across these diverse feature spaces to evaluate model performance and use this alignment to guide training. Experiments on benchmark datasets with varying amounts of labels under both synthetic and real-world noise demonstrate that CLID-MU outperforms state-of-the-art methods. The code is released at https://github.com/ruofanhu/CLID-MU.

Problem

• Meta-learning without clean labeled dataset
• Handling noisy labels in deep learning
• Cross-layer consistency for robust training

Innovation

• Uses cross-layer information divergence
• Evaluates model via feature space alignment
• No need for clean labeled meta-dataset