FedCoT: Federated Chain-of-Thought Distillation for Large Language Models

2024-06-18

Conference on Empirical Methods in Natural Language Processing

AI Summary

To address the dual challenges of user data privacy preservation and small language model (SLM) performance enhancement in resource-constrained large language model (LLM) deployment, this paper proposes FedCoT—a novel federated chain-of-thought (CoT) knowledge distillation framework. FedCoT enables the transfer of LLMs’ CoT reasoning capabilities to client-side SLMs without sharing raw user data. It introduces a dual-mechanism differential privacy strategy: an exponential mechanism safeguards prompt privacy, while an adaptive exponential mechanism dynamically balances privacy budget allocation and CoT chain utility. Extensive experiments on multi-text generation tasks demonstrate significant SLM performance gains, with empirical validation confirming simultaneous optimization of privacy guarantees and generation quality. The implementation is publicly released as part of the FATE-LLM project.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Safety and Robustness
• Planning, Routing, and Scheduling: Planning with Language Models

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Large Language Models (LLMs) have emerged as a transformative force in artificial intelligence, demonstrating exceptional proficiency across various tasks. However, their deployment in resource-constrained environments and concerns over user data privacy pose significant challenges. In contrast, Small Language Models (SLMs) offer computational efficiency but often lag in performance. To address these issues, we propose FedCoT, a federated framework designed for the Chain-of-Thought (CoT) distillation of knowledge from LLMs to SLMs, while ensuring the preservation of clients'data privacy. FedCoT ensures secure and efficient knowledge transfer from an LLM on a high-powered server to an SLM on a resource-constrained client, while adhering to privacy requirements. Leveraging perturbed prompts and rationales generated through the CoT approach, the framework enhances the performance of the client's SLM without compromising user data privacy within a multi-task learning framework. We propose two privacy protection strategies: the Exponential Mechanism Strategy and the Adaptive Exponential Mechanism Strategy, which balance user prompt privacy and the usability of rationales. Empirical evaluation on various text generation tasks demonstrates the effectiveness of FedCoT in training task-specific SLMs with enhanced performance while prioritizing data privacy protection. Our code has been contributed to the FATE open-source project and is now publicly accessible at extit{https://github.com/FederatedAI/FATE-LLM/tree/main/python/fate_llm/algo/fedcot}

Problem

• Enhancing Small Language Models' performance while preserving data privacy
• Transferring knowledge from Large to Small Language Models securely
• Balancing prompt privacy and rationale usability in federated learning

Innovation

• Federated framework for Chain-of-Thought distillation
• Privacy protection via perturbed prompts and rationales
• Knowledge transfer from large to small language models