Online Convex Optimisation: The Optimal Switching Regret for all Segmentations Simultaneously

2024-05-31

Neural Information Processing Systems

AI Summary

This paper studies switching regret minimization in online convex optimization under non-stationary environments—i.e., achieving asymptotically optimal performance across *all* possible time-segment partitions. Existing methods fail to simultaneously accommodate arbitrary partition structures and adapt to unknown dynamic change rates. To address this, we propose the first algorithm that is both *partition-agnostic* and *dynamically adaptive* to the underlying variation rate. Our approach builds upon a hierarchical expert framework, integrating adaptive learning rates with a logarithmic-complexity tree-based structure for efficient expert selection and maintenance. Theoretically, our algorithm attains a switching regret of $O(sqrt{S T log T})$ for any $S$-segment partition, while requiring only $O(log T)$ time and space complexity per round—substantially improving upon prior bounds. Moreover, we establish a novel dynamic regret upper bound that explicitly matches the true variation rate, marking the first such result in the literature.

Technology Category

• Machine Learning: Online Learning & Bandits
• Search and Optimization: Non-convex Optimization
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments

Abstract

We consider the classic problem of online convex optimisation. Whereas the notion of static regret is relevant for stationary problems, the notion of switching regret is more appropriate for non-stationary problems. A switching regret is defined relative to any segmentation of the trial sequence, and is equal to the sum of the static regrets of each segment. In this paper we show that, perhaps surprisingly, we can achieve the asymptotically optimal switching regret on every possible segmentation simultaneously. Our algorithm for doing so is very efficient: having a space and per-trial time complexity that is logarithmic in the time-horizon. Our algorithm also obtains novel bounds on its dynamic regret: being adaptive to variations in the rate of change of the comparator sequence.

Problem

• Achieving optimal switching regret for all segmentations simultaneously.
• Developing an efficient algorithm with logarithmic space and time complexity.
• Obtaining adaptive bounds on dynamic regret for varying comparator sequences.

Innovation

• Achieves optimal switching regret simultaneously
• Efficient logarithmic space-time complexity
• Adaptive dynamic regret bounds