LiST: Lipschitz Scaling Training for Robust and Calibrated Neural Networks

2026-07-08

AI Summary

Existing methods struggle to simultaneously achieve high accuracy, robustness, and calibration in neural networks. This work proposes Lipschitz Scaling Training (LiST), which establishes, for the first time, a theoretical connection between Lipschitz constraints and temperature scaling. By dynamically adjusting the global Lipschitz constant during training, LiST embeds calibration directly into the learning process, automatically identifying a calibration-optimal operating point along the accuracy–robustness Pareto frontier. The method integrates margin-aware Lipschitz constraints, dynamic constant adaptation, and calibration-aware optimization, and further improves sample efficiency by reusing calibration data after convergence. Experiments on CIFAR-10, CIFAR-100, and Tiny-ImageNet demonstrate that LiST matches baseline performance in both accuracy and robustness while achieving well-calibrated predictions without any post-hoc processing.

Abstract

While accuracy, robustness, and calibration are all essential for reliable neural networks, they are often studied separately; developing models that satisfy all three simultaneously remains a central challenge. Lipschitz-constrained models guarantee robustness by design, yet the manual selection of the Lipschitz constraint L governs the resulting accuracy-robustness trade-off, and their calibration properties remain largely underexplored. In this work, we highlight a theoretical and empirical link between the enforced Lipschitz constraint and Temperature Scaling, a state-of-the-art calibration method. Specifically, we find that for a given training scheme, there exists a non-trivial value L* that yields an out-of-the-box calibrated network, and that calibration acts as a principled criterion to select a well-defined operating point on the accuracy-robustness Pareto front. Leveraging these insights, we introduce Lipschitz Scaling Training (LiST), a novel training paradigm that iteratively adjusts the global Lipschitz constant to reach this operating point. Through a margin parameter in the training loss, LiST further enables the construction of a fully calibrated Pareto front, allowing users to navigate the accuracy-robustness trade-off while remaining calibrated throughout. At convergence, LiST also enables the reintegration of calibration data into training, improving sample efficiency without sacrificing calibration. We validate LiST on CIFAR-10/100 and Tiny-ImageNet, demonstrating competitive accuracy and robustness against constrained and unconstrained baselines, while remaining calibrated out of the box. Code is available at GitHub.

Problem

• accuracy
• robustness
• calibration
• Lipschitz constraint
• Pareto front

Innovation

• Lipschitz constraint
• calibration
• robustness
• accuracy-robustness trade-off
• Temperature Scaling