Safe and Efficient Online Convex Optimization with Linear Budget Constraints and Partial Feedback

2024-12-05

arXiv.org

AI Summary

This paper studies online convex optimization under unknown linear budget constraints, where only bandit feedback is available—i.e., gradients of the objective and constraint functions are inaccessible, and only constraint function values are observed. To address this partial-information setting, we propose the Safe and Efficient Lyapunov Optimization (SELO) algorithm—the first to simultaneously achieve $O(sqrt{T})$ adaptive regret and zero cumulative constraint violation. Methodologically, SELO integrates a Lyapunov stability framework with primal-dual updates: it solves a strongly convex, smooth unconstrained surrogate problem for efficient primal iteration; employs simple gradient-based dual updates; and explicitly leverages a Lyapunov function to guide safe exploration. We establish theoretical guarantees of dual optimality in both regret and constraint satisfaction. Empirical evaluation on distributed data center energy-aware scheduling confirms SELO’s safety, computational efficiency, and practical effectiveness.

Technology Category

• Search and Optimization: Non-convex Optimization
• Constraint Satisfaction and Optimization: Constraint Optimization
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

This paper studies online convex optimization with unknown linear budget constraints, where only the gradient information of the objective and the bandit feedback of constraint functions are observed. We propose a safe and efficient Lyapunov-optimization algorithm (SELO) that can achieve an $O(sqrt{T})$ regret and zero cumulative constraint violation. The result also implies SELO achieves $O(sqrt{T})$ regret when the budget is hard and not allowed to be violated. The proposed algorithm is computationally efficient as it resembles a primal-dual algorithm where the primal problem is an unconstrained, strongly convex and smooth problem, and the dual problem has a simple gradient-type update. The algorithm and theory are further justified in a simulated application of energy-efficient task processing in distributed data centers.

Problem

• Online convex optimization with unknown linear budget constraints.
• Achieves O(sqrt(T)) regret and zero cumulative constraint violation.
• Applied in energy-efficient task processing in distributed data centers.

Innovation

• Lyapunov-optimization algorithm for OCO
• Achieves O(sqrt(T)) regret, zero violation
• Efficient primal-dual approach for constraints