Learning from Noisy Labels with Contrastive Co-Transformer

2025-03-04

AI Summary

Deep learning models often suffer from overfitting and poor generalization under label noise. To address this, we propose Co-Transformer—a novel architecture integrating contrastive learning with co-training. It is the first method to embed contrastive learning into a co-training framework and leverages the inherent robustness of Vision Transformers to label noise, enabling end-to-end joint optimization over both clean and noisy samples—without label cleaning, sample selection, or label correction. The model is trained via unified optimization of a joint contrastive loss and classification loss, yielding a simple, computationally efficient design. Evaluated on six standard noisy-label benchmarks—including Clothing1M—Co-Transformer consistently outperforms state-of-the-art methods, achieving significant average accuracy gains. These results demonstrate its superior generalization capability and training stability under realistic label noise conditions.

Technology Category

• Computer Vision: Adversarial Attacks & Robustness
• Machine Learning: Multi-class/Multi-label Learning & Extreme Classification
• Natural Language Processing: Safety and Robustness

Application Category

• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Deep learning with noisy labels is an interesting challenge in weakly supervised learning. Despite their significant learning capacity, CNNs have a tendency to overfit in the presence of samples with noisy labels. Alleviating this issue, the well known Co-Training framework is used as a fundamental basis for our work. In this paper, we introduce a Contrastive Co-Transformer framework, which is simple and fast, yet able to improve the performance by a large margin compared to the state-of-the-art approaches. We argue the robustness of transformers when dealing with label noise. Our Contrastive Co-Transformer approach is able to utilize all samples in the dataset, irrespective of whether they are clean or noisy. Transformers are trained by a combination of contrastive loss and classification loss. Extensive experimental results on corrupted data from six standard benchmark datasets including Clothing1M, demonstrate that our Contrastive Co-Transformer is superior to existing state-of-the-art methods.

Problem

• Addresses overfitting in CNNs with noisy labels.
• Proposes Contrastive Co-Transformer for robust learning.
• Improves performance on noisy datasets using transformers.

Innovation

• Contrastive Co-Transformer for noisy labels
• Combines contrastive and classification loss
• Utilizes all samples, clean or noisy