A space-decoupling framework for optimization on bounded-rank matrices with orthogonally invariant constraints

2025-01-23

AI Summary

Optimization of finite-rank matrices under orthogonal invariance constraints exhibits geometric complexity due to coupled structural constraints, hindering efficient Riemannian optimization. Method: We propose a spatial decoupling framework that separates the rank constraint and orthogonal invariance constraint into two independent spaces, enabling construction of a smooth manifold amenable to standard Riemannian optimization algorithms. Contribution/Results: We establish, for the first time, that the tangent cone of the coupled constraint set decomposes as the intersection of the individual tangent cones; we rigorously prove equivalence between the decoupled formulation and the original problem; and we introduce a general, geometrically transparent paradigm for low-rank constrained optimization. Empirically, the method significantly improves convergence speed and solution accuracy across diverse applications—including spherical fitting, graph similarity measurement, low-rank semidefinite programming, Markov model order reduction, and reinforcement learning.

Technology Category

• Search and Optimization: Non-convex Optimization
• Constraint Satisfaction and Optimization: Constraint Optimization
• Machine Learning: Matrix & Tensor Methods

Application Category

• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

Imposing additional constraints on low-rank optimization has garnered growing interest. However, the geometry of coupled constraints hampers the well-developed low-rank structure and makes the problem intricate. To this end, we propose a space-decoupling framework for optimization on bounded-rank matrices with orthogonally invariant constraints. The ``space-decoupling"is reflected in several ways. We show that the tangent cone of coupled constraints is the intersection of tangent cones of each constraint. Moreover, we decouple the intertwined bounded-rank and orthogonally invariant constraints into two spaces, leading to optimization on a smooth manifold. Implementing Riemannian algorithms on this manifold is painless as long as the geometry of additional constraints is known. In addition, we unveil the equivalence between the reformulated problem and the original problem. Numerical experiments on real-world applications -- spherical data fitting, graph similarity measuring, low-rank SDP, model reduction of Markov processes, reinforcement learning, and deep learning -- validate the superiority of the proposed framework.

Problem

• Orthogonally Invariant Constraints
• Finite Rank Matrix Optimization
• Coupled Constraints

Innovation

• Space-Decoupling Framework
• Bounded-Rank Matrices
• Riemannian Algorithms