Non-Stationary Functional Bilevel Optimization

2026-01-21

AI Summary

Existing function-space bilevel optimization methods struggle to handle online non-stationary environments. This work proposes SmoothFBO, the first algorithm to extend function-space bilevel optimization to such settings. By employing a time-smoothed stochastic hypergradient estimator, SmoothFBO reduces variance in gradient estimates, enabling stable outer-loop updates and achieving sublinear regret. The method offers strong theoretical guarantees and scalability, while naturally encompassing classical parametric bilevel optimization as a special case. Empirical evaluations on non-stationary hyperparameter optimization and model-based reinforcement learning tasks demonstrate that SmoothFBO significantly outperforms existing approaches, confirming its effectiveness and broad applicability.

Technology Category

• Reasoning under Uncertainty: Stochastic Optimization
• Machine Learning: Online Learning & Bandits
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Social networks and social learning
• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments

Abstract

Functional bilevel optimization (FBO) provides a powerful framework for hierarchical learning in function spaces, yet current methods are limited to static offline settings and perform suboptimally in online, non-stationary scenarios. We propose SmoothFBO, the first algorithm for non-stationary FBO with both theoretical guarantees and practical scalability. SmoothFBO introduces a time-smoothed stochastic hypergradient estimator that reduces variance through a window parameter, enabling stable outer-loop updates with sublinear regret. Importantly, the classical parametric bilevel case is a special reduction of our framework, making SmoothFBO a natural extension to online, non-stationary settings. Empirically, SmoothFBO consistently outperforms existing FBO methods in non-stationary hyperparameter optimization and model-based reinforcement learning, demonstrating its practical effectiveness. Together, these results establish SmoothFBO as a general, theoretically grounded, and practically viable foundation for bilevel optimization in online, non-stationary scenarios.

Problem

• Non-Stationary
• Functional Bilevel Optimization
• Online Optimization
• Hyperparameter Optimization
• Stochastic Hypergradient

Innovation

• non-stationary
• functional bilevel optimization
• time-smoothed hypergradient
• online learning
• sublinear regret