Communication-Pipelined Split Federated Learning for Foundation Model Fine-Tuning in UAV Networks

2025-11-19

AI Summary

To address straggler-dominated training and communication resource underutilization—caused by parallel gradient transmission—in split federated learning (SFL) for fine-tuning foundation models over UAV networks, this paper proposes a communication-pipelined split federated learning framework. Methodologically, it introduces, for the first time, serial gradient transmission coupled with downlink-first scheduling and intra-round asynchronous training to mitigate tail latency. Furthermore, it designs an attention-based deep reinforcement learning algorithm that jointly optimizes model partitioning points, uplink bandwidth allocation, and server computation frequency, while incorporating dynamic UAV trajectory planning for real-time decision-making. Experimental results demonstrate that the proposed approach significantly reduces per-round training latency and energy consumption compared to parallel gradient transmission baselines and ablation variants, closely approaching the performance of optimal static partitioning—thereby enabling efficient, low-overhead distributed fine-tuning.

Technology Category

• Machine Learning: Distributed Machine Learning & Federated Learning
• Planning, Routing, and Scheduling: Planning with Language Models
• Search and Optimization: Learning to Search

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Systems and Infrastructure for Web, Mobile and WoT: Federated Web and WoT systems, including distributed, federated and edge-based data processing

Abstract

Deploying foundation models (FMs) on uncrewed aerial vehicles (UAVs) promises broad ``low-altitude economy''applications. Split federated learning (SFL)-based fine-tuning leverages distributed data while keeping raw data local and reduces client-side burden by partitioning the model between client and server. However, the per-round training latency is dominated by stragglers. Training paradigms featuring parallel gradient transmission (GT) allocate dedicated portions of downlink communication resources to each client. They may leave resources idle and suffer from prolonged GT latency, especially in UAV networks, where the communication latency typically far exceeds the computation latency. To address this, we propose a sequential GT paradigm, where the server dedicates all downlink resources for the current GT. We further propose communication-pipelined SFL (CPSFL), characterized by downlink GT priority scheduling and intra-round asynchronous training. We investigate CPSFL-based LoRA fine-tuning of FMs in UAV networks and formulate an optimization problem to minimize a weighted sum of per-round training latency and worst-case client energy consumption by optimizing the split point selection (SPS) and the computing and communication resource allocation (CCRA) (the uplink bandwidth allocation and the server computing frequency allocation). To solve this problem, we develop an attention-based deep reinforcement learning (DRL) framework, where the base station agent decides the split point and the CCRA in each round by leveraging previous round information, including UAV trajectories. Simulation results show that the proposed DRL-based CPSFL scheme outperforms the parallel GT benchmarks, the ablation variants, the fixed CCRA scheme, while approaching the best fixed-SPS scheme.

Problem

• Reduces training latency caused by stragglers in UAV networks
• Optimizes resource allocation for split federated learning fine-tuning
• Minimizes energy consumption while training foundation models on drones

Innovation

• Sequential gradient transmission paradigm for UAV networks
• Communication-pipelined SFL with priority scheduling
• Attention-based DRL framework for resource optimization