Sobolev norm inconsistency of kernel interpolation

2025-04-29

AI Summary

This paper investigates generalization consistency of minimum-norm interpolation in bounded-kernel reproducing kernel Hilbert spaces (RKHS), focusing on generalization error measured under a continuous scale of Sobolev-type norms—interpolating between the $L^2$ and RKHS norms. Using spectral analysis, Sobolev embedding theorems, and precise characterization of kernel eigenvalue decay rates, the authors derive the first tight lower bound on the generalization error within this norm scale. They rigorously prove that when the Sobolev smoothness index exceeds a threshold determined solely by the RKHS embedding exponent and the kernel’s eigenvalue decay rate, the interpolation solution is necessarily inconsistent. This result systematically reveals an intrinsic inconsistency of kernel interpolation across the continuous norm spectrum, providing a novel theoretical perspective and a rigorous criterion for understanding the generalization limits of overparameterized interpolation models.

Technology Category

• Machine Learning: Kernel Methods
• Reasoning under Uncertainty: Stochastic Optimization
• Search and Optimization: Non-convex Optimization

Application Category

• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization

Abstract

We study the consistency of minimum-norm interpolation in reproducing kernel Hilbert spaces corresponding to bounded kernels. Our main result give lower bounds for the generalization error of the kernel interpolation measured in a continuous scale of norms that interpolate between $L^2$ and the hypothesis space. These lower bounds imply that kernel interpolation is always inconsistent, when the smoothness index of the norm is larger than a constant that depends only on the embedding index of the hypothesis space and the decay rate of the eigenvalues.

Problem

• Study consistency of kernel interpolation in bounded kernel spaces
• Provide lower bounds for generalization error in interpolated norms
• Show kernel inconsistency when smoothness exceeds certain threshold

Innovation

• Minimum-norm interpolation in kernel Hilbert spaces
• Lower bounds for generalization error in norms
• Inconsistency when smoothness exceeds threshold