Gradient Rectification for Robust Calibration under Distribution Shift

2025-08-27

AI Summary

Deep neural networks often exhibit overconfident predictions under distribution shifts, severely compromising reliability in safety-critical applications. Existing calibration methods require access to target-domain data—rendering them impractical in real-world deployment scenarios. To address this, we propose the first target-domain-agnostic calibration framework grounded in the frequency domain. We theoretically and empirically reveal, for the first time, how distribution shifts degrade calibration performance through the lens of frequency spectra. Our method introduces a low-frequency filtering module to strengthen reliance on domain-invariant features, and incorporates a gradient correction constraint that jointly preserves in-distribution and out-of-distribution calibration during optimization. Extensive experiments on diverse synthetic and real-world shift benchmarks—including CIFAR-10/100-C and WILDS—demonstrate significant improvements in calibration metrics (e.g., ECE) without sacrificing in-distribution calibration accuracy. The proposed approach establishes a robust, practical, and unified calibration paradigm.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Computer Vision: Adversarial Attacks & Robustness
• Natural Language Processing: Safety and Robustness

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Security and Privacy: Large-scale security measurements
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems

Abstract

Deep neural networks often produce overconfident predictions, undermining their reliability in safety-critical applications. This miscalibration is further exacerbated under distribution shift, where test data deviates from the training distribution due to environmental or acquisition changes. While existing approaches improve calibration through training-time regularization or post-hoc adjustment, their reliance on access to or simulation of target domains limits their practicality in real-world scenarios. In this paper, we propose a novel calibration framework that operates without access to target domain information. From a frequency-domain perspective, we identify that distribution shifts often distort high-frequency visual cues exploited by deep models, and introduce a low-frequency filtering strategy to encourage reliance on domain-invariant features. However, such information loss may degrade In-Distribution (ID) calibration performance. Therefore, we further propose a gradient-based rectification mechanism that enforces ID calibration as a hard constraint during optimization. Experiments on synthetic and real-world shifted datasets, including CIFAR-10/100-C and WILDS, demonstrate that our method significantly improves calibration under distribution shift while maintaining strong in-distribution performance.

Problem

• Addresses overconfident predictions in neural networks
• Improves calibration under distribution shift conditions
• Eliminates need for target domain information access

Innovation

• Low-frequency filtering for domain-invariant features
• Gradient-based rectification mechanism for calibration
• No target domain information required framework