Decentralized Federated Learning with Model Caching on Mobile Agents

2024-08-26

arXiv.org

AI Summary

To address slow convergence and low accuracy in decentralized federated learning (DFL) caused by intermittent communication in mobile agent networks, this paper proposes a cache-enhanced distributed collaborative training framework. Our key contributions are: (1) the first delay-tolerant model caching mechanism, enabling asynchronous model propagation and aggregation; (2) a staleness-aware aggregation scheme, with theoretical analysis of how cached model delays impact convergence; and (3) an adaptive caching strategy tailored to dynamic network topologies. Evaluated on a vehicular network simulation, our method accelerates convergence by up to 2.1× compared to baseline DFL and improves test accuracy by 6.8% over a cache-free DFL baseline. These results demonstrate its effectiveness and robustness under high mobility and low connectivity conditions.

Technology Category

• Machine Learning: Distributed Machine Learning & Federated Learning
• Multiagent Systems: Agent Communication
• Search and Optimization: Distributed Search

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Federated Learning (FL) trains a shared model using data and computation power on distributed agents coordinated by a central server. Decentralized FL (DFL) utilizes local model exchange and aggregation between agents to reduce the communication and computation overheads on the central server. However, when agents are mobile, the communication opportunity between agents can be sporadic, largely hindering the convergence and accuracy of DFL. In this paper, we propose Cached Decentralized Federated Learning (Cached-DFL) to investigate delay-tolerant model spreading and aggregation enabled by model caching on mobile agents. Each agent stores not only its own model, but also models of agents encountered in the recent past. When two agents meet, they exchange their own models as well as the cached models. Local model aggregation utilizes all models stored in the cache. We theoretically analyze the convergence of Cached-DFL, explicitly taking into account the model staleness introduced by caching. We design and compare different model caching algorithms for different DFL and mobility scenarios. We conduct detailed case studies in a vehicular network to systematically investigate the interplay between agent mobility, cache staleness, and model convergence. In our experiments, Cached-DFL converges quickly, and significantly outperforms DFL without caching.

Problem

• Enhances Decentralized Federated Learning efficiency
• Reduces communication overhead in mobile environments
• Improves model convergence with caching strategies

Innovation

• Decentralized Federated Learning
• Model Caching on Mobile Agents
• Delay-Tolerant Model Aggregation