Compressed Proximal Federated Learning for Non-Convex Composite Optimization on Heterogeneous Data

2026-03-08

AI Summary

This work proposes FedCEF, an algorithm designed to address key challenges in federated composite optimization under heterogeneous data, including low communication efficiency, difficulty in handling non-smooth regularizers, and convergence instability caused by compression bias. FedCEF decouples proximal updates from communication, enabling clients to handle non-smooth terms locally, and introduces a downlink strategy that allows exact reconstruction of the global control variable without explicit transmission. Furthermore, it incorporates an error feedback mechanism with control variates to effectively mitigate the adverse effects of quantization noise and data heterogeneity. Theoretical analysis establishes that FedCEF converges to a controllable residual error under non-convex settings, and experiments demonstrate its ability to maintain high accuracy even at an extreme 1% compression rate, significantly reducing communication overhead.

Technology Category

• Search and Optimization: Non-convex Optimization
• Machine Learning: Distributed Machine Learning & Federated Learning
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Systems and Infrastructure for Web, Mobile and WoT: Federated Web and WoT systems, including distributed, federated and edge-based data processing
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Federated Composite Optimization (FCO) has emerged as a promising framework for training models with structural constraints (e.g., sparsity) in distributed edge networks. However, simultaneously achieving communication efficiency and convergence robustness remains a significant challenge, particularly when dealing with non-smooth regularizers, statistical heterogeneity, and the restrictions of biased compression. To address these issues, we propose FedCEF (Federated Composite Error Feedback), a novel algorithm tailored for non-convex FCO. FedCEF introduces a decoupled proximal update scheme that separates the proximal operator from communication, enabling clients to handle non-smooth terms locally while transmitting compressed information. To mitigate the noise from aggressive quantization and the bias from non-IID data, FedCEF integrates a rigorous error feedback mechanism with control variates. Furthermore, we design a communication-efficient pre-proximal downlink strategy that allows clients to exactly reconstruct global control variables without explicit transmission. We theoretically establish that FedCEF achieves sublinear convergence to a bounded residual error under general non-convexity, which is controllable via the step size and batch size. Extensive experiments on real datasets validate FedCEF maintains competitive model accuracy even under extreme compression ratios (e.g., 1%), significantly reducing the total communication volume compared to uncompressed baselines.

Problem

• Federated Composite Optimization
• Non-Convex Optimization
• Communication Efficiency
• Statistical Heterogeneity
• Biased Compression

Innovation

• Federated Composite Optimization
• Error Feedback
• Proximal Decoupling
• Communication Compression
• Non-Convex Optimization