Stability of Primal-Dual Gradient Flow Dynamics for Multi-Block Convex Optimization Problems

2024-08-28

arXiv.org

AI Summary

This work investigates the stability of primal-dual gradient flow dynamics for multi-block composite convex optimization under generalized consensus constraints, particularly targeting large-scale distributed settings involving multiple nonsmooth terms. We propose a continuous-time dynamical framework based on the proximal augmented Lagrangian and establish global exponential convergence via Lyapunov analysis. Compared with mainstream discrete-time algorithms such as ADMM and EXTRA, our approach significantly relaxes standard assumptions—namely, strong convexity and smoothness of objective components, as well as algebraic connectivity of the communication graph—and further proves the necessity of certain relaxed conditions. The theoretical results provide milder, more broadly applicable convergence guarantees for distributed nonsmooth optimization. Numerical experiments demonstrate the efficiency and practicality of the proposed dynamics in both parallel and distributed implementations.

Technology Category

• Search and Optimization: Non-convex Optimization
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Security and Privacy: Large-scale security measurements
• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments

Abstract

We examine stability properties of primal-dual gradient flow dynamics for composite convex optimization problems with multiple, possibly nonsmooth, terms in the objective function under the generalized consensus constraint. The proposed dynamics are based on the proximal augmented Lagrangian and they provide a viable alternative to ADMM which faces significant challenges from both analysis and implementation viewpoints in large-scale multi-block scenarios. In contrast to customized algorithms with individualized convergence guarantees, we provide a systematic approach for solving a broad class of challenging composite optimization problems. We leverage various structural properties to establish global (exponential) convergence guarantees for the proposed dynamics. Our assumptions are much weaker than those required to prove (exponential) stability of various primal-dual dynamics as well as (linear) convergence of discrete-time methods, e.g., standard two-block and multi-block ADMM and EXTRA algorithms. Finally, we show necessity of some of our structural assumptions for exponential stability and provide computational experiments to demonstrate the convenience of the proposed dynamics for parallel and distributed computing applications.

Problem

• Analyze stability of primal-dual gradient flow for multi-block convex optimization
• Propose alternative to ADMM for large-scale nonsmooth composite optimization
• Establish global convergence guarantees under weaker structural assumptions

Innovation

• Proximal augmented Lagrangian dynamics
• Systematic approach for composite optimization
• Global exponential convergence guarantees