Implicit Riemannian Optimism with Applications to Min-Max Problems

2025-01-30

AI Summary

This work addresses online optimization and minimax problems on Hadamard manifolds. We propose the first curvature-independent implicit Riemannian optimistic online learning algorithm, supporting intrinsic manifold constraints. Our method employs implicit iterative updates and a novel Riemannian optimistic gradient design, achieving the optimal Euclidean regret bound of $O(sqrt{T})$ without requiring any geometric constants—such as curvature lower bounds—for the first time. We further extend the framework to $g$-convex–$g$-concave smooth minimax optimization, attaining a gradient complexity of $ ilde{O}(1/varepsilon^2)$, which nearly matches the theoretical lower bound and significantly improves upon existing manifold-based algorithms. Key contributions include: (i) eliminating curvature dependence entirely; (ii) unifying treatment of online learning and minimax optimization on manifolds; and (iii) recovering Euclidean optimality in nonlinear geometric spaces.

Technology Category

• Machine Learning: Learning with Manifolds
• Search and Optimization: Non-convex Optimization
• Computer Vision: Learning & Optimization for CV

Application Category

• Responsible Web: Human-perceived consequences of algorithmic deployment on the web
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems

Abstract

We introduce a Riemannian optimistic online learning algorithm for Hadamard manifolds based on inexact implicit updates. Unlike prior work, our method can handle in-manifold constraints, and matches the best known regret bounds in the Euclidean setting with no dependence on geometric constants, like the minimum curvature. Building on this, we develop algorithms for g-convex, g-concave smooth min-max problems on Hadamard manifolds. Notably, one method nearly matches the gradient oracle complexity of the lower bound for Euclidean problems, for the first time.

Problem

• Hadamard manifolds
• optimal solutions
• algorithm performance

Innovation

• Hadamard Manifold
• Optimization Algorithm
• Robust Performance