Mix from Failure: Confusion-Pairing Mixup for Long-Tailed Recognition

2024-11-12

arXiv.org

AI Summary

Long-tailed image recognition suffers from insufficient samples and high inter-class confusion for tail classes, leading to biased decision boundaries. To address this, we propose a real-time data augmentation method grounded in the model’s confusion distribution, introducing Confusion-Pair Mixing (CP-Mix)—the first approach to dynamically sample highly confusable class pairs via online estimation of inter-class confusion and perform semantically consistent sample mixing using a debiased Mixup formulation. This enhances diversity among minority-class samples while rectifying fragile decision boundaries. Crucially, CP-Mix operates purely at the data level, requiring no architectural modifications or loss-function redesign. Extensive experiments on ImageNet-LT, iNaturalist, and CIFAR-LT demonstrate that our method significantly outperforms state-of-the-art long-tailed learning approaches: tail-class accuracy improves markedly, and confusion rates decrease substantially—validating its effectiveness in mitigating classifier confusion and boundary shift.

Technology Category

• Machine Learning: Mixture of Experts (MoE)
• Computer Vision: Adversarial Attacks & Robustness
• Search and Optimization: Mixed Discrete/Continuous Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Long-tailed image recognition is a computer vision problem considering a real-world class distribution rather than an artificial uniform. Existing methods typically detour the problem by i) adjusting a loss function, ii) decoupling classifier learning, or iii) proposing a new multi-head architecture called experts. In this paper, we tackle the problem from a different perspective to augment a training dataset to enhance the sample diversity of minority classes. Specifically, our method, namely Confusion-Pairing Mixup (CP-Mix), estimates the confusion distribution of the model and handles the data deficiency problem by augmenting samples from confusion pairs in real-time. In this way, CP-Mix trains the model to mitigate its weakness and distinguish a pair of classes it frequently misclassifies. In addition, CP-Mix utilizes a novel mixup formulation to handle the bias in decision boundaries that originated from the imbalanced dataset. Extensive experiments demonstrate that CP-Mix outperforms existing methods for long-tailed image recognition and successfully relieves the confusion of the classifier.

Problem

• Addresses long-tailed image recognition with real-world class distributions.
• Enhances minority class diversity via Confusion-Pairing Mixup (CP-Mix).
• Mitigates classifier confusion and decision boundary bias in imbalanced datasets.

Innovation

• Augments training data using confusion pairs
• Estimates model confusion for real-time augmentation
• Novel mixup formulation reduces decision boundary bias