Minimax Adaptive Online Nonparametric Regression over Besov Spaces

2025-05-26

AI Summary

This paper studies online adversarial regression in Besov spaces $B_{pq}^s$ with unknown smoothness parameters $(s,p,q)$, where $s > d/p$ and $1 le p,q le infty$. To address the challenges posed by spatially heterogeneous and highly irregular function regularity, we propose the first parameter-free adaptive wavelet algorithm, integrating multiresolution analysis, local thresholding estimation, and online convex optimization. Furthermore, we design a spatio-temporal jointly local adaptive mechanism that enables dynamic tracking of non-uniform smoothness—achieved for the first time in this setting. Theoretically, the global variant achieves the minimax-optimal regret bound over $B_{pq}^s$, while the locally adaptive variant significantly outperforms the global approach in heterogeneous environments; its regret bound explicitly reflects regional variations in regularity.

Technology Category

• Machine Learning: Online Learning & Bandits
• Search and Optimization: Adversarial Search
• Game Theory and Economic Paradigms: Adversarial Learning

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

We study online adversarial regression with convex losses against a rich class of continuous yet highly irregular prediction rules, modeled by Besov spaces $B_{pq}^s$ with general parameters $1 leq p,q leq infty$ and smoothness $s>d/p$. We introduce an adaptive wavelet-based algorithm that performs sequential prediction without prior knowledge of $(s,p,q)$, and establish minimax-optimal regret bounds against any comparator in $B_{pq}^s$. We further design a locally adaptive extension capable of dynamically tracking spatially inhomogeneous smoothness. This adaptive mechanism adjusts the resolution of the predictions over both time and space, yielding refined regret bounds in terms of local regularity. Consequently, in heterogeneous environments, our adaptive guarantees can significantly surpass those obtained by standard global methods.

Problem

• Online adversarial regression in Besov spaces
• Adaptive prediction without prior smoothness knowledge
• Dynamic tracking of spatially inhomogeneous smoothness

Innovation

• Wavelet-based adaptive online regression algorithm
• Minimax-optimal regret without prior parameters
• Locally adaptive tracking of spatial inhomogeneity