A Survey on Split Learning for LLM Fine-Tuning: Models, Systems, and Privacy Optimizations

2026-04-27

AI Summary

This work addresses the dual challenges of prohibitive computational costs and privacy risks associated with sensitive data in large language model (LLM) fine-tuning, which hinder adoption by resource-constrained organizations. It presents the first systematic survey of split learning approaches for LLM fine-tuning, proposing a unified fine-grained training framework and establishing a structured taxonomy across three dimensions: model optimization, system efficiency, and privacy preservation. By synthesizing advances in model partitioning strategies, system-level optimizations, and privacy attack–defense mechanisms, this study clarifies the technical landscape, introduces a standardized evaluation benchmark, and provides both theoretical foundations and practical guidance for scalable, robust, and secure collaborative model adaptation.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Search and Optimization: Learning to Search

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Fine-tuning unlocks large language models (LLMs) for specialized applications, but its high computational cost often puts it out of reach for resource-constrained organizations. While cloud platforms could provide the needed resources, data privacy concerns make sharing sensitive information with third parties risky. A promising solution is split learning for LLM fine-tuning, which divides the model between clients and a server, allowing collaborative and secure training through exchanged intermediate data, thus enabling resource-constrained participants to adapt LLMs safely. % In light of this, a growing body of literature has emerged to advance this paradigm, introducing varied model methods, system optimizations, and privacy defense-attack techniques for split learning. To bring clarity and direction to the field, a comprehensive survey is needed to classify, compare, and critique these diverse approaches. This paper fills the gap by presenting the first extensive survey dedicated to split learning for LLM fine-tuning. We propose a unified, fine-grained training pipeline to pinpoint key operational components and conduct a systematic review of state-of-the-art work across three core dimensions: model-level optimization, system-level efficiency, and privacy preservation. Through this structured taxonomy, we establish a foundation for advancing scalable, robust, and secure collaborative LLM adaptation.

Problem

• LLM fine-tuning
• split learning
• data privacy
• resource constraints
• collaborative training

Innovation

• Split Learning
• LLM Fine-Tuning
• Privacy Preservation
• Model Optimization
• System Efficiency