Can Differentially Private Fine-tuning LLMs Protect Against Privacy Attacks?

2025-04-28

AI Summary

The practical privacy guarantees of differential privacy (DP) in fine-tuning large language models (LLMs) remain poorly understood. Method: This work systematically evaluates DP’s effectiveness against data extraction and membership inference attacks across three fine-tuning paradigms—LoRA, full-parameter fine-tuning, and Adapter—under controlled privacy budgets ε. Contribution/Results: We demonstrate for the first time that the choice of fine-tuning method fundamentally governs the DP privacy–utility trade-off: moderate ε values suffice to substantially reduce privacy risk, whereas full-parameter fine-tuning suffers from utility collapse and is thus unsuitable for DP. We quantitatively characterize the privacy–utility curves for each method, empirically confirming that DP meaningfully mitigates training data leakage. Based on these findings, we propose a principled, privacy-aware guideline for selecting fine-tuning methods in sensitive applications.

Technology Category

• Machine Learning: Privacy
• Natural Language Processing: Safety and Robustness
• Computer Vision: Diffusion Models for Vision

Application Category

• Security and Privacy: Large-scale security measurements
• User Modeling, Personalization and Recommendation: User privacy protection in personalized systems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Fine-tuning large language models (LLMs) has become an essential strategy for adapting them to specialized tasks; however, this process introduces significant privacy challenges, as sensitive training data may be inadvertently memorized and exposed. Although differential privacy (DP) offers strong theoretical guarantees against such leakage, its empirical privacy effectiveness on LLMs remains unclear, especially under different fine-tuning methods. In this paper, we systematically investigate the impact of DP across fine-tuning methods and privacy budgets, using both data extraction and membership inference attacks to assess empirical privacy risks. Our main findings are as follows: (1) Differential privacy reduces model utility, but its impact varies significantly across different fine-tuning methods. (2) Without DP, the privacy risks of models fine-tuned with different approaches differ considerably. (3) When DP is applied, even a relatively high privacy budget can substantially lower privacy risk. (4) The privacy-utility trade-off under DP training differs greatly among fine-tuning methods, with some methods being unsuitable for DP due to severe utility degradation. Our results provide practical guidance for privacy-conscious deployment of LLMs and pave the way for future research on optimizing the privacy-utility trade-off in fine-tuning methodologies.

Problem

• Assessing DP's effectiveness in protecting LLM fine-tuning privacy
• Comparing privacy risks across different fine-tuning methods with DP
• Evaluating privacy-utility trade-offs in DP-applied LLM fine-tuning

Innovation

• Differential privacy applied to LLM fine-tuning
• Privacy-utility trade-off varies by fine-tuning method
• High privacy budget reduces risk significantly