Primal-Dual Methods for Nonsmooth Nonconvex Optimization with Orthogonality Constraints

2026-04-05

AI Summary

This work addresses the challenges of poor scalability, limited parallelizability, and complex subproblems in nonsmooth nonconvex optimization with orthogonality constraints by proposing a retraction-free primal-dual linearized smoothed augmented Lagrangian method. The proposed algorithm introduces, for the first time, a retraction-free primal-dual framework to orthogonality-constrained optimization, eliminating nested loops and intricate subproblem solvers in favor of a single-loop iteration scheme. Leveraging the Kurdyka–Łojasiewicz property, the method is theoretically shown to converge to an $\varepsilon$-KKT point with an iteration complexity of $O(\varepsilon^{-3})$, without requiring Riemannian retractions. Numerical experiments demonstrate that the algorithm significantly outperforms existing approaches in both computational efficiency and scalability.

Abstract

Recent advancements in data science have significantly elevated the importance of orthogonally constrained optimization problems. The Riemannian approach has become a popular technique for addressing these problems due to the advantageous computational and analytical properties of the Stiefel manifold. Nonetheless, the interplay of nonsmoothness alongside orthogonality constraints introduces substantial challenges to current Riemannian methods, including scalability, parallelizability, complicated subproblems, and cumulative numerical errors that threaten feasibility. In this paper, we take a retraction-free primal-dual approach and propose a linearized smoothing augmented Lagrangian method specifically designed for nonsmooth and nonconvex optimization with orthogonality constraints. Our proposed method is single-loop and free of subproblem solving. We establish its iteration complexity of $O(ε^{-3})$ for finding $ε$-KKT points, matching the best-known results in the Riemannian optimization literature. Additionally, by invoking the standard Kurdyka-Lojasiewicz (KL) property, we demonstrate asymptotic sequential convergence of the proposed algorithm. Numerical experiments on both smooth and nonsmooth orthogonal constrained problems demonstrate the superior computational efficiency and scalability of the proposed method compared with state-of-the-art algorithms.

Problem

• nonsmooth optimization
• nonconvex optimization
• orthogonality constraints
• Stiefel manifold
• primal-dual methods

Innovation

• primal-dual method
• orthogonality constraints
• nonsmooth nonconvex optimization
• augmented Lagrangian
• retraction-free