Learning from Imperfect Text Guidance: Robust Long-Tail Visual Recognition with High-Noise Label

2026-04-24

AI Summary

Real-world visual recognition often confronts the dual challenges of long-tailed class distributions and high label noise, which degrade model performance due to label-image mismatches. This work proposes a novel approach that leverages auxiliary textual information through pretrained vision-language models to generate weak teacher supervision (WTS) signals, exploiting their cross-modal alignment capabilities. By dynamically comparing original noisy labels with text-derived predictions, the method adaptively activates WTS to correct inconsistencies. Extensive experiments demonstrate significant performance gains on both synthetic and real-world long-tailed datasets under high label noise, with particularly strong robustness in extreme noise conditions. The proposed framework establishes a new paradigm for long-tailed learning with noisy labels by effectively integrating multimodal semantic guidance.

Technology Category

• Machine Learning: Multi-class/Multi-label Learning & Extreme Classification
• Computer Vision: Language and Vision
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Real-world data often exhibit long-tailed distributions with numerous noisy labels, substantially degrading the performance of deep models. While prior research has made progress in addressing this combined challenge, it overlooks the severe label-image mismatch inherent to high-noise settings, thereby limiting their effectiveness. Given that observed labels, though mismatched with images, still retain category information, we propose employing auxiliary text information from labels to address label-image inconsistencies in long-tailed noisy data. Specifically, we leverage the intrinsic cross-modal alignment in pre-trained visual-language models to correct the label-image inconsistencies. This supervisory signal, referred to as Weak Teacher Supervision (WTS), is unaffected by label noise and data distribution biases, albeit exhibits limited accuracy. Therefore, the activation of WTS is determined by evaluating the discrepancy between text-predicted labels and observed labels. Extensive experiments demonstrate the superior performance of WTS across synthetic and real-world datasets, particularly under high-noise conditions. The source code is available at https://anonymous.4open.science/r/WTS-0F3C.

Problem

• long-tail visual recognition
• noisy labels
• label-image mismatch
• high-noise settings
• real-world data

Innovation

• Weak Teacher Supervision
• visual-language models
• long-tailed recognition
• noisy labels
• cross-modal alignment