CalibrateMix: Guided-Mixup Calibration of Image Semi-Supervised Models

2025-11-16

AI Summary

In semi-supervised learning (SSL), models often suffer from poor calibration due to overconfident pseudo-labels, leading to a mismatch between predicted confidence and actual accuracy. To address this, we propose a training-dynamics-guided Mixup calibration method: by analyzing the loss evolution of labeled and unlabeled samples during training, our approach adaptively identifies easy-to-learn and hard-to-learn instances and applies differentiated Mixup augmentation to suppress calibration bias introduced by pseudo-labeling. The method requires no additional annotations or architectural modifications. It preserves strong SSL classification performance while substantially improving predictive reliability. On benchmark datasets—including CIFAR-10, CIFAR-100, and SVHN—our method reduces Expected Calibration Error (ECE) by an average of 32.7% and achieves higher classification accuracy than state-of-the-art SSL methods (e.g., FixMatch, UDA). To our knowledge, this is the first SSL framework that systematically reconciles high accuracy with high calibration fidelity.

Technology Category

• Machine Learning: Semi-Supervised Learning
• Computer Vision: Adversarial Attacks & Robustness
• Reasoning under Uncertainty: Relational Probabilistic Models

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems

Abstract

Semi-supervised learning (SSL) has demonstrated high performance in image classification tasks by effectively utilizing both labeled and unlabeled data. However, existing SSL methods often suffer from poor calibration, with models yielding overconfident predictions that misrepresent actual prediction likelihoods. Recently, neural networks trained with { t mixup} that linearly interpolates random examples from the training set have shown better calibration in supervised settings. However, calibration of neural models remains under-explored in semi-supervised settings. Although effective in supervised model calibration, random mixup of pseudolabels in SSL presents challenges due to the overconfidence and unreliability of pseudolabels. In this work, we introduce CalibrateMix, a targeted mixup-based approach that aims to improve the calibration of SSL models while maintaining or even improving their classification accuracy. Our method leverages training dynamics of labeled and unlabeled samples to identify ``easy-to-learn'' and ``hard-to-learn'' samples, which in turn are utilized in a targeted mixup of easy and hard samples. Experimental results across several benchmark image datasets show that our method achieves lower expected calibration error (ECE) and superior accuracy compared to existing SSL approaches.

Problem

• Addresses poor calibration in semi-supervised image classification models
• Mitigates overconfidence from unreliable pseudolabels in mixup techniques
• Improves model calibration accuracy while maintaining classification performance

Innovation

• Targeted mixup of easy and hard samples
• Leveraging training dynamics for sample identification
• Improving calibration while maintaining classification accuracy