Towards Understanding The Calibration Benefits of Sharpness-Aware Minimization

2025-05-29

AI Summary

Deep neural networks often exhibit poor calibration—overconfident predictions—in safety-critical applications such as medical diagnosis and autonomous driving, posing significant reliability risks. This paper systematically investigates the intrinsic mechanism by which Sharpness-Aware Minimization (SAM) improves model calibration, revealing for the first time that SAM implicitly maximizes the entropy of the predictive distribution, thereby achieving automatic calibration. Building on this insight, we propose Calibrated SAM (CSAM), an enhanced variant that preserves model accuracy while substantially improving calibration performance. Extensive experiments across benchmarks—including ImageNet-1K—demonstrate that CSAM consistently outperforms both standard SAM and state-of-the-art calibration methods, achieving significant reductions in Expected Calibration Error (ECE) and Brier Score. Our work provides a novel theoretical perspective on the interplay between optimization algorithms and model uncertainty, and delivers a practical, deployment-ready solution for trustworthy AI in safety-critical domains.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Computer Vision: Adversarial Attacks & Robustness
• Natural Language Processing: Safety and Robustness

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Security and Privacy: Large-scale security measurements
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Deep neural networks have been increasingly used in safety-critical applications such as medical diagnosis and autonomous driving. However, many studies suggest that they are prone to being poorly calibrated and have a propensity for overconfidence, which may have disastrous consequences. In this paper, unlike standard training such as stochastic gradient descent, we show that the recently proposed sharpness-aware minimization (SAM) counteracts this tendency towards overconfidence. The theoretical analysis suggests that SAM allows us to learn models that are already well-calibrated by implicitly maximizing the entropy of the predictive distribution. Inspired by this finding, we further propose a variant of SAM, coined as CSAM, to ameliorate model calibration. Extensive experiments on various datasets, including ImageNet-1K, demonstrate the benefits of SAM in reducing calibration error. Meanwhile, CSAM performs even better than SAM and consistently achieves lower calibration error than other approaches

Problem

• SAM reduces overconfidence in deep neural networks
• CSAM improves model calibration better than SAM
• Maximizing predictive entropy enhances calibration in models

Innovation

• Uses sharpness-aware minimization (SAM)
• Implicitly maximizes predictive entropy
• Proposes CSAM for better calibration