Personalized Wireless Federated Learning for Large Language Models

2024-04-20

arXiv.org

AI Summary

To address privacy leakage, data heterogeneity, and high communication overhead in large language model (LLM) federated fine-tuning over wireless networks, this paper proposes two lightweight personalized federated fine-tuning frameworks: PFIT (Reinforcement Learning–Driven Personalized Instruction Tuning) and PFTT (Global Adapter + Local LoRA Collaborative Tuning). Both frameworks eliminate global parameter aggregation, enabling end-to-end personalized modeling. PFIT employs reinforcement learning to dynamically optimize instruction-tuning policies, while PFTT hierarchically decouples global knowledge and local characteristics via a hybrid adapter architecture. Experimental results demonstrate that, compared to baseline methods, the proposed approaches accelerate convergence, improve personalized task accuracy by 18.7%, and reduce communication overhead by 62%. Collectively, they achieve a superior trade-off among privacy preservation, model personalization, and wireless resource efficiency.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Planning, Routing, and Scheduling: Planning with Language Models

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Systems and Infrastructure for Web, Mobile and WoT: Federated Web and WoT systems, including distributed, federated and edge-based data processing
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Large Language Models (LLMs) have revolutionized natural language processing tasks. However, their deployment in wireless networks still face challenges, i.e., a lack of privacy and security protection mechanisms. Federated Learning (FL) has emerged as a promising approach to address these challenges. Yet, it suffers from issues including inefficient handling with big and heterogeneous data, resource-intensive training, and high communication overhead. To tackle these issues, we first compare different learning stages and their features of LLMs in wireless networks. Next, we introduce two personalized wireless federated fine-tuning methods with low communication overhead, i.e., (1) Personalized Federated Instruction Tuning (PFIT), which employs reinforcement learning to fine-tune local LLMs with diverse reward models to achieve personalization; (2) Personalized Federated Task Tuning (PFTT), which can leverage global adapters and local Low-Rank Adaptations (LoRA) to collaboratively fine-tune local LLMs, where the local LoRAs can be applied to achieve personalization without aggregation. Finally, we perform simulations to demonstrate the effectiveness of the proposed two methods and comprehensively discuss open issues.

Problem

• Addresses security and privacy challenges in LLM training within wireless networks
• Reduces energy consumption and communication delay in federated learning for LLMs
• Implements personalized learning and alignment for stable FL processes

Innovation

• Adapter and LoRA reduce energy consumption
• Global partial aggregation cuts communication delay
• Personalized loss function enables customized learning