An Inexact Halpern Iteration with Application to Distributionally Robust Optimization

2024-02-08

arXiv.org

AI Summary

This work addresses the monotone inclusion problem—arising, e.g., in Wasserstein distributionally robust optimization (DRO) with convex–concave minimax structures—by proposing novel deterministic and stochastic variants of the inexact Halpern iteration. Departing from standard inexactness criteria, we introduce the inexact Halpern method systematically within the stochastic first-order framework for the first time and develop a unified convergence analysis. Theoretically, both settings achieve the optimal $O(k^{-1})$ rate in residual norm convergence. Experiments demonstrate that the method efficiently solves two classes of data-driven DRO problems, exhibiting strong empirical performance and competitiveness on nonlinear convex–concave losses and stochastic robust learning tasks. Key contributions include: (i) the first extension of inexact Halpern iteration to stochastic monotone inclusions and distributionally robust learning; and (ii) simultaneous attainment of theoretical optimality and practical efficacy.

Technology Category

• Reasoning under Uncertainty: Stochastic Optimization
• Intelligent Robots: Learning & Optimization for ROB
• Machine Learning: Adversarial Learning & Robustness

Application Category

• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

The Halpern iteration for solving monotone inclusion problems has gained increasing interests in recent years due to its simple form and appealing convergence properties. In this paper, we investigate the inexact variants of the scheme in both deterministic and stochastic settings. We conduct extensive convergence analysis and show that by choosing the inexactness tolerances appropriately, the inexact schemes admit an $O(k^{-1})$ convergence rate in terms of the (expected) residue norm. Our results relax the state-of-the-art inexactness conditions employed in the literature while sharing the same competitive convergence properties. We then demonstrate how the proposed methods can be applied for solving two classes of data-driven Wasserstein distributionally robust optimization problems that admit convex-concave min-max optimization reformulations. We highlight its capability of performing inexact computations for distributionally robust learning with stochastic first-order methods and for general nonlinear convex-concave loss functions, which are competitive in the literature.

Problem

• Investigates inexact Halpern iteration for monotone inclusion problems
• Applies method to Wasserstein distributionally robust optimization problems
• Enables inexact computations for robust learning with stochastic methods

Innovation

• Inexact Halpern iteration for robust optimization
• Convergence rate O(k^{-1}) with relaxed conditions
• Stochastic first-order methods for learning