Federated Communication-Efficient Multi-Objective Optimization

2024-10-21

arXiv.org

AI Summary

This work addresses three key challenges in multi-objective optimization (MOO) for federated learning (FL): excessive communication overhead, limited convergence analysis under realistic assumptions, and inadequate preference modeling. To this end, we propose FedCMOO—a communication-efficient MOO algorithm for FL. FedCMOO is the first to achieve target-count-agnostic gradient compression, enabling collaborative optimization of multiple non-convex objectives within a single shared model. It introduces a preference-guided weighted update mechanism, allowing clients to explicitly specify desired trade-offs among objectives. Moreover, its convergence analysis is established under milder non-convex smoothness assumptions than prior works. Extensive experiments on image classification and recommendation tasks demonstrate that FedCMOO significantly improves Pareto front quality, reduces total communication volume by 62%, and decreases convergence error by 37%, consistently outperforming state-of-the-art baselines.

Technology Category

• Machine Learning: Distributed Machine Learning & Federated Learning
• Search and Optimization: Non-convex Optimization
• Constraint Satisfaction and Optimization: Constraint Optimization

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Systems and Infrastructure for Web, Mobile and WoT: Federated Web and WoT systems, including distributed, federated and edge-based data processing
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs

Abstract

We study a federated version of multi-objective optimization (MOO), where a single model is trained to optimize multiple objective functions. MOO has been extensively studied in the centralized setting but is less explored in federated or distributed settings. We propose FedCMOO, a novel communication-efficient federated multi-objective optimization (FMOO) algorithm that improves the error convergence performance of the model compared to existing approaches. Unlike prior works, the communication cost of FedCMOO does not scale with the number of objectives, as each client sends a single aggregated gradient to the central server. We provide a convergence analysis of the proposed method for smooth and non-convex objective functions under milder assumptions than in prior work. In addition, we introduce a variant of FedCMOO that allows users to specify a preference over the objectives in terms of a desired ratio of the final objective values. Through extensive experiments, we demonstrate the superiority of our proposed method over baseline approaches.

Problem

• Federated multi-objective optimization with communication efficiency
• Improving error convergence in distributed MOO settings
• Enabling client-specified preference ratios for objective values

Innovation

• Federated multi-objective optimization algorithm
• Single aggregated gradient reduces communication
• Supports user-specified objective preferences