CED-EF: Compressed Exact Diffusion with Error Feedback for Multi-Agent Learning

2026-08-24

AI Summary

本文提出CED-EF方法，通过误差反馈和压缩通信解决多智能体学习中的去中心化随机优化问题，改善了对压缩级别和网络条件的依赖。

Abstract

We study decentralized stochastic optimization over a network of $N$ agents under compressed communication. We propose CED-EF, an exact diffusion-based method with error feedback that directly accommodates biased $δ$-contractive compressors while communicating one compressed model-sized vector per node per iteration. For smooth nonconvex objectives with unbiased stochastic gradients whose variance is bounded by $σ^2$, where $σ\geq0$, we establish a convergence rate whose leading stochastic term is $\mathcal O(σ/\sqrt{NK})$. For $σ>0$, the dominant dependence of the corresponding transient time on the number of agents, compression level, and spectral gap $Δ_λ$ is $\mathcal O(N^3/(δ^4Δ_λ^4))$, with fixed problem-dependent factors suppressed. Under the Polyak--Łojasiewicz condition, CED-EF attains a leading stochastic term $\widetilde{\mathcal O}(σ^2/(NK))$ with transient time on the order of $\widetilde{\mathcal O}(N/(δ^2Δ_λ^2))$. These dependencies improve the compression and/or network dependence of existing results. Numerical experiments on least-squares and logistic-regression problems illustrate the performance advantages of CED-EF.

Problem

• Decentralized Stochastic Optimization
• Compressed Communication
• Multi-Agent Learning

Innovation

• Compressed Exact Diffusion
• Error Feedback
• Biased δ-contractive Compressors
• Nonconvex Optimization
• Decentralized Learning