PostHoc FREE Calibrating on Kolmogorov Arnold Networks

2025-03-03

AI Summary

Kolmogorov–Arnold networks (KANs) exhibit miscalibration—overconfidence in dense input regions and underconfidence in sparse ones. This work systematically characterizes the intrinsic relationship between KAN calibration performance and key architectural hyperparameters, including layer width and grid order—the first such analysis. To address miscalibration, we propose Temperature Scaling Loss (TSL), an end-to-end trainable calibration objective that embeds a learnable temperature parameter directly into the training loss, eliminating the need for post-hoc calibration. Leveraging B-spline parameterization, we validate TSL on CIFAR-10, CIFAR-100, and Tiny-ImageNet. Results show that TSL significantly reduces expected calibration error (ECE) and other calibration metrics, consistently outperforming standard temperature scaling and Platt scaling, while incurring no additional inference overhead. Our method enhances both the reliability and practical utility of KANs’ probabilistic predictions.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Relational Probabilistic Models
• Computer Vision: Learning & Optimization for CV

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Kolmogorov Arnold Networks (KANs) are neural architectures inspired by the Kolmogorov Arnold representation theorem that leverage B Spline parameterizations for flexible, locally adaptive function approximation. Although KANs can capture complex nonlinearities beyond those modeled by standard MultiLayer Perceptrons (MLPs), they frequently exhibit miscalibrated confidence estimates manifesting as overconfidence in dense data regions and underconfidence in sparse areas. In this work, we systematically examine the impact of four critical hyperparameters including Layer Width, Grid Order, Shortcut Function, and Grid Range on the calibration of KANs. Furthermore, we introduce a novel TemperatureScaled Loss (TSL) that integrates a temperature parameter directly into the training objective, dynamically adjusting the predictive distribution during learning. Both theoretical analysis and extensive empirical evaluations on standard benchmarks demonstrate that TSL significantly reduces calibration errors, thereby improving the reliability of probabilistic predictions. Overall, our study provides actionable insights into the design of spline based neural networks and establishes TSL as a robust loss solution for enhancing calibration.

Problem

• Addresses miscalibration in Kolmogorov Arnold Networks (KANs).
• Examines hyperparameters affecting KANs' calibration performance.
• Introduces TemperatureScaled Loss to improve predictive reliability.

Innovation

• TemperatureScaled Loss for calibration improvement
• B Spline parameterizations in KANs
• Dynamic adjustment of predictive distribution