A regret minimization approach to fixed-point iterations

2025-09-25

AI Summary

This work addresses fixed-point computation for non-self-mappings. We systematically transform online regret minimization algorithms—particularly Online Gradient Descent (OGD) and AdaGrad-type methods—into fixed-point iteration schemes with provable convergence guarantees. The proposed framework generalizes the classical Krasnoselskii–Mann iteration and, for the first time, introduces regret analysis from online optimization to derive adaptive fixed-point updates in the non-self-mapping setting. Theoretically, under standard assumptions (e.g., cocoercivity or monotonicity), our methods achieve strong convergence and exhibit explicit iteration complexity bounds. Empirically, the AdaGrad-based adaptive scheme significantly outperforms the standard Krasnoselskii–Mann method on challenging nonsmooth and nonmonotone fixed-point problems—including variational inequalities and implicit deep network solving—demonstrating both accelerated convergence and enhanced robustness.

Technology Category

• Search and Optimization: Non-convex Optimization
• Machine Learning: Online Learning & Bandits
• Game Theory and Economic Paradigms: Adversarial Learning

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

We propose a conversion scheme that turns regret minimizing algorithms into fixed point iterations, with convergence guarantees following from regret bounds. The resulting iterations can be seen as a grand extension of the classical Krasnoselskii--Mann iterations, as the latter are recovered by converting the Online Gradient Descent algorithm. This approach yields new simple iterations for finding fixed points of non-self operators. We also focus on converting algorithms from the AdaGrad family of regret minimizers, and thus obtain fixed point iterations with adaptive guarantees of a new kind. Numerical experiments on various problems demonstrate faster convergence of AdaGrad-based fixed point iterations over Krasnoselskii--Mann iterations.

Problem

• Converting regret minimization algorithms into fixed-point iterations with convergence guarantees
• Extending classical Krasnoselskii-Mann iterations using Online Gradient Descent
• Developing adaptive fixed-point iterations via AdaGrad for faster convergence

Innovation

• Converting regret minimization algorithms into fixed-point iterations
• Extending Krasnoselskii-Mann iterations via Online Gradient Descent
• Developing adaptive fixed-point iterations using AdaGrad family