Decentralized Stochastic Subgradient-type Methods with Communication Compression for Nonsmooth Nonconvex Optimization

2026-07-02

AI Summary

This work addresses decentralized non-smooth non-convex optimization under communication constraints by proposing a unified algorithmic framework that integrates unbiased or contractive compression (with error compensation), stochastic subgradient updates, gradient-tracking momentum, and sign-based regularization. For the first time, global convergence is rigorously established under conditions where the objective function is non-smooth and fails to satisfy Clarke regularity, leveraging differential inclusion theory to analyze consensus errors and averaged trajectories. Both theoretical analysis and empirical experiments demonstrate that the proposed method significantly reduces communication overhead while effectively preserving optimization accuracy, thereby achieving a favorable trade-off between communication efficiency and convergence performance.

Abstract

In this paper, we consider the nonsmooth nonconvex decentralized optimization problem, where inter-agent communication is compressed. We propose a general framework that unifies various decentralized stochastic subgradient-type methods with unbiased compression and contractive compression with error compensation. By relating the consensus-error iterates and the averaged iterates to the trajectories of continuous-time differential inclusions, we establish global convergence for all methods encompassed by our framework when the objective functions are nonsmooth and lack Clarke regularity. Based on our framework, we further develop several compression-based methods, including decentralized stochastic subgradient methods utilizing sign-based regularization and gradient-tracking momentum. Preliminary numerical experiments empirically support our theoretical results and highlight the communication-accuracy trade-off of the newly developed methods.

Problem

• decentralized optimization
• nonsmooth nonconvex
• communication compression
• stochastic subgradient

Innovation

• decentralized optimization
• communication compression
• nonsmooth nonconvex optimization
• stochastic subgradient methods
• error compensation