Mirror Descent on Riemannian Manifolds

2026-03-18

AI Summary

Optimization over large-scale Riemannian manifolds poses significant challenges for the direct application of classical mirror descent methods. This work proposes a Riemannian Mirror Descent (RMD) framework that generalizes mirror descent to arbitrary Riemannian manifolds through reparameterization and develops its stochastic variant. For the first time, non-asymptotic convergence guarantees are established for mirror descent on general Riemannian manifolds. When specialized to the Stiefel manifold, the framework naturally yields Riemannian gradient descent and its stochastic extension. The theoretical analysis integrates tools from Riemannian geometry, stochastic gradient estimation, and non-convex optimization. Empirical evaluations demonstrate the efficiency and scalability of the proposed methods across diverse tasks, including image processing, policy optimization, and neural network training.

Technology Category

• Machine Learning: Learning with Manifolds
• Search and Optimization: Non-convex Optimization
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Mirror Descent (MD) is a scalable first-order method widely used in large-scale optimization, with applications in image processing, policy optimization, and neural network training. This paper generalizes MD to optimization on Riemannian manifolds. In particular, we develop a Riemannian Mirror Descent (RMD) framework via reparameterization and further propose a stochastic variant of RMD. We also establish non-asymptotic convergence guarantees for both RMD and stochastic RMD. As an application to the Stiefel manifold, our RMD framework reduces to the Curvilinear Gradient Descent (CGD) method proposed in [26]. Moreover, when specializing the stochastic RMD framework to the Stiefel setting, we obtain a stochastic extension of CGD, which effectively addresses large-scale manifold optimization problems.

Problem

• Mirror Descent
• Riemannian Manifolds
• Manifold Optimization
• Stiefel Manifold
• Large-scale Optimization

Innovation

• Riemannian Mirror Descent
• stochastic optimization
• manifold optimization
• non-asymptotic convergence
• Stiefel manifold