On the Rate of Convergence of Kolmogorov-Arnold Network Regression Estimators

2025-09-24

AI Summary

This work addresses the suboptimal convergence rate of Kolmogorov–Arnold networks (KANs) in multivariate function approximation. Methodologically, it constructs univariate activation functions via B-splines and systematically analyzes the approximation properties of both additive and compositional KANs within the Sobolev space framework, deriving an optimal rule for selecting the number of B-spline knots. Theoretically, it establishes the first nonparametric regression theory for KANs, rigorously proving that both architectures achieve the minimax-optimal convergence rate $O(n^{-2r/(2r+1)})$ for functions in the Sobolev space $W^{r,infty}([0,1]^d)$. This resolves a critical gap in the theoretical understanding of KANs’ approximation capability. Numerical experiments corroborate the theoretical predictions. Collectively, this work furnishes a rigorous approximation-theoretic foundation for structured neural networks and provides interpretable, theoretically grounded guidelines for hyperparameter design.

Technology Category

• Machine Learning: Kernel Methods
• Search and Optimization: Non-convex Optimization
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Search and Retrieval-Augmented AI: Web query analysis, representation and understanding
• Web Mining and Content Analysis: Models for Web evolution

Abstract

Kolmogorov-Arnold Networks (KANs) offer a structured and interpretable framework for multivariate function approximation by composing univariate transformations through additive or multiplicative aggregation. This paper establishes theoretical convergence guarantees for KANs when the univariate components are represented by B-splines. We prove that both additive and hybrid additive-multiplicative KANs attain the minimax-optimal convergence rate $O(n^{-2r/(2r+1)})$ for functions in Sobolev spaces of smoothness $r$. We further derive guidelines for selecting the optimal number of knots in the B-splines. The theory is supported by simulation studies that confirm the predicted convergence rates. These results provide a theoretical foundation for using KANs in nonparametric regression and highlight their potential as a structured alternative to existing methods.

Problem

• Establishing theoretical convergence guarantees for Kolmogorov-Arnold Networks
• Proving KANs achieve minimax-optimal convergence rates in Sobolev spaces
• Providing theoretical foundation for KANs in nonparametric regression applications

Innovation

• KANs use B-splines for univariate transformations
• Achieve minimax-optimal convergence rate O(n^(-2r/(2r+1)))
• Provide guidelines for optimal B-spline knot selection