Wahkon: A Statistically Principled Deep RKHS Superposition Network

2026-05-13

AI Summary

This work addresses the lack of finite-sample statistical guarantees and reliable uncertainty calibration in deep learning, alongside the poor performance of classical reproducing kernel Hilbert space (RKHS) methods in high dimensions. By integrating the Kolmogorov superposition theorem with RKHS regularization, the authors propose a novel deep network architecture featuring explicit control over inter-layer complexity. They establish a finite-dimensional deep representation theorem, extending the duality between splines and Gaussian processes to deep structures and revealing fundamental theoretical trade-offs among depth, width, and function regularity. Empirical evaluations on synthetic benchmarks and single-cell CITE-seq data demonstrate that the proposed method significantly outperforms multilayer perceptrons, neural tangent kernels, and Kolmogorov–Arnold networks, achieving high accuracy while offering interpretability and rigorous statistical guarantees.

Abstract

Deep learning excels at prediction but often lacks finite-sample guarantees and calibrated uncertainty; RKHS (Reproducing Kernel Hilbert Space)-based methods provide those guarantees but struggle to adapt in high dimensions. We propose Wahkon, a deep RKHS superposition network that unifies Kolmogorov's superposition principle with RKHS regularization in the smoothing-spline tradition of Wahba. This yields a finite-dimensional deep representer theorem that makes training tractable and provides explicit layerwise complexity control. We show the penalized estimator is exactly the MAP (maximum a posteriori) estimate under a hierarchical Gaussian-process prior, extending the spline/GP duality to deep compositions. Using metric-entropy arguments, we establish minimax-optimal convergence rates under mild smoothness and clarify how depth and width trade off with regularity. Empirically, Wahkon outperforms multilayer perceptrons, Neural Tangent Kernels, and Kolmogorov--Arnold Networks across simulation benchmarks and a single-cell CITE-seq study. By unifying Kolmogorov's superposition principle with RKHS regularization, Wahkon delivers accuracy, interpretability, and statistical rigor in a single framework.

Problem

• deep learning
• RKHS
• uncertainty quantification
• finite-sample guarantees
• high-dimensional adaptation

Innovation

• RKHS
• Kolmogorov superposition
• deep representer theorem
• Gaussian process
• minimax convergence