VFLAIR-LLM: A Comprehensive Framework and Benchmark for Split Learning of LLMs

2025-08-05

AI Summary

To address the security challenges of adapting large language models (LLMs) in privacy-sensitive and resource-constrained edge environments, this paper proposes VFLAIR-LLM—the first lightweight, scalable split learning (SL) framework specifically designed for LLMs. It enables privacy-preserving inference and fine-tuning across diverse model partitioning strategies (e.g., layer-wise or module-level splitting), NLP tasks, and heterogeneous datasets. We introduce the first systematic SL-LLM benchmark, integrating five representative privacy attacks and nine defense mechanisms, along with practical guidelines for partition configuration, defense selection, and hyperparameter tuning. Extensive experiments demonstrate VFLAIR-LLM’s effectiveness and robustness under stringent computational constraints. Our framework establishes a reproducible, extensible technical paradigm for deploying LLMs in privacy-critical settings.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Computer Vision: Large Vision Models

Application Category

• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Security and Privacy: Security and privacy of machine learning and AI applications

Abstract

With the advancement of Large Language Models (LLMs), LLM applications have expanded into a growing number of fields. However, users with data privacy concerns face limitations in directly utilizing LLM APIs, while private deployments incur significant computational demands. This creates a substantial challenge in achieving secure LLM adaptation under constrained local resources. To address this issue, collaborative learning methods, such as Split Learning (SL), offer a resource-efficient and privacy-preserving solution for adapting LLMs to private domains. In this study, we introduce VFLAIR-LLM (available at https://github.com/FLAIR-THU/VFLAIR-LLM), an extensible and lightweight split learning framework for LLMs, enabling privacy-preserving LLM inference and fine-tuning in resource-constrained environments. Our library provides two LLM partition settings, supporting three task types and 18 datasets. In addition, we provide standard modules for implementing and evaluating attacks and defenses. We benchmark 5 attacks and 9 defenses under various Split Learning for LLM(SL-LLM) settings, offering concrete insights and recommendations on the choice of model partition configurations, defense strategies, and relevant hyperparameters for real-world applications.

Problem

• Secure LLM adaptation under constrained local resources
• Privacy-preserving LLM inference and fine-tuning
• Resource-efficient collaborative learning for LLMs

Innovation

• Extensible split learning framework for LLMs
• Privacy-preserving inference and fine-tuning
• Benchmarks attacks and defenses in SL-LLM