Kernel-based guarantees for nonlinear parametric models in Bayesian optimization

2026-05-13

AI Summary

Existing Bayesian optimization methods lack theoretical guarantees for adaptive data acquisition in nonlinearly parameterized models. This work proposes an analytical framework based on the reproducing kernel Hilbert space (RKHS) induced by kernels over the parameter space, integrated with regularized convex loss minimization, to establish a unified confidence bound theory for widely used nonlinear surrogate models. For the first time, this framework provides rigorous convergence guarantees for nonlinearly parameterized models under adaptive sampling, enabling a variety of novel acquisition strategies—including stochastic regularization and randomized model maximization—and substantially broadening the theoretical applicability of Bayesian optimization.

Abstract

Modern Bayesian optimization and adaptive sampling methods increasingly rely on nonlinear parametric models, yet theoretical guarantees for such models under adaptive data collection remain limited. Existing analyses largely focus on Gaussian processes, kernel machines, linear models, or linearized neural approximations, leaving a gap between theory and the nonlinear models used in practice. We develop a kernel based framework for analyzing regularized nonlinear parametric models trained on adaptively collected data. Our approach uses kernels over the parameter space to induce reproducing kernel Hilbert space structures over the corresponding model class, yielding confidence bounds for models trained with broad classes of regularized convex losses. We show how these bounds can support convergence guarantees for nonlinear acquisition and surrogate models, including randomized regularized policies that select points by maximizing a trained random model. These results provide a unified route to analyzing nonlinear parametric models in Bayesian optimization and related adaptive optimization settings.

Problem

• nonlinear parametric models
• Bayesian optimization
• adaptive sampling
• theoretical guarantees
• kernel methods

Innovation

• kernel-based analysis
• nonlinear parametric models
• Bayesian optimization
• adaptive sampling
• reproducing kernel Hilbert space