🤖 AI Summary
This paper studies vector-valued online kernel regression under noise: sequentially approximating an unknown regression function (f_mu) in a reproducing kernel Hilbert space (RKHS) with prior smoothness assumptions, given i.i.d. noisy observations ((omega_m, y_m) sim mu). We propose a novel vector-valued online learning algorithm that updates estimates sample-by-sample. Our main contribution is the first non-asymptotic convergence rate bound—namely, (mathbb{E}| hat{f}_m - f_mu |_{mathcal{H}}^2 leq C^2 (m+1)^{-s/(2+s)})—for the expected squared RKHS norm error. This bound explicitly quantifies the interplay between the target function’s smoothness parameter (s) and the convergence rate. Unlike prior asymptotic or problem-specific analyses, our result provides the first non-asymptotic theoretical guarantee for vector-valued online learning with an explicit, interpretable exponent. The analysis integrates tools from vector-valued kernel methods, online optimization, functional analysis, and concentration inequalities.
📝 Abstract
We consider the problem of approximating the regression function from noisy vector-valued data by an online learning algorithm using an appropriate reproducing kernel Hilbert space (RKHS) as prior. In an online algorithm, i.i.d. samples become available one by one by a random process and are successively processed to build approximations to the regression function. We are interested in the asymptotic performance of such online approximation algorithms and show that the expected squared error in the RKHS norm can be bounded by $C^2 (m+1)^{-s/(2+s)}$, where $m$ is the current number of processed data, the parameter $0<sleq 1$ expresses an additional smoothness assumption on the regression function and the constant $C$ depends on the variance of the input noise, the smoothness of the regression function and further parameters of the algorithm.
