Decentralized Nonsmooth Nonconvex Optimization with Client Sampling

2026-01-27

AI Summary

This work addresses non-smooth non-convex optimization over decentralized networks, where each node’s objective function is Lipschitz continuous. The authors propose an efficient stochastic first-order method that incorporates client sampling and leverages spectral gap analysis of the mixing matrix to significantly reduce communication overhead while maintaining optimal sample complexity. The approach is further extended to the zeroth-order setting. Theoretical analysis shows that the algorithm reaches a $(\delta,\varepsilon)$-Goldstein stationary point with $\mathcal{O}(\delta^{-1}\varepsilon^{-3})$ sample and computational complexity, and achieves a communication complexity of $\tilde{\mathcal{O}}(\gamma^{-1/2}\delta^{-1}\varepsilon^{-3})$, outperforming existing methods. Experimental results confirm the algorithm’s effectiveness and superiority in practice.

Technology Category

• Search and Optimization: Non-convex Optimization
• Reasoning under Uncertainty: Stochastic Optimization
• Machine Learning: Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments

Abstract

This paper considers decentralized nonsmooth nonconvex optimization problem with Lipschitz continuous local functions. We propose an efficient stochastic first-order method with client sampling, achieving the $(\delta,\epsilon)$-Goldstein stationary point with the overall sample complexity of ${\mathcal O}(\delta^{-1}\epsilon^{-3})$, the computation rounds of ${\mathcal O}(\delta^{-1}\epsilon^{-3})$, and the communication rounds of ${\tilde{\mathcal O}}(\gamma^{-1/2}\delta^{-1}\epsilon^{-3})$, where $\gamma$ is the spectral gap of the mixing matrix for the network. Our results achieve the optimal sample complexity and the sharper communication complexity than existing methods. We also extend our ideas to zeroth-order optimization. Moreover, the numerical experiments show the empirical advantage of our methods.

Problem

• decentralized optimization
• nonsmooth nonconvex
• client sampling
• stochastic optimization
• communication complexity

Innovation

• decentralized optimization
• nonsmooth nonconvex optimization
• client sampling
• Goldstein stationary point
• zeroth-order optimization