Adaptive Clipping for Privacy-Preserving Few-Shot Learning: Enhancing Generalization with Limited Data

2025-03-27

AI Summary

To address the significant degradation in generalization performance of few-shot meta-learning under differential privacy (DP), this paper proposes a dynamic adaptive gradient clipping mechanism—the first to enable fine-grained privacy-utility co-optimization *during training* in DP-MAML and DP-Reptile frameworks. The method jointly incorporates sensitivity-aware threshold adaptation and generalization regularization, ensuring strict $(varepsilon,delta)$-DP while enhancing model robustness. Evaluated on multiple few-shot benchmark datasets, it achieves an average accuracy improvement of 5.2% and reduces utility loss by 37% under identical privacy budgets, establishing the state-of-the-art privacy-utility trade-off. This work provides a scalable, privacy-preserving meta-learning paradigm for settings characterized by both privacy sensitivity and label scarcity.

Technology Category

• Machine Learning: Privacy
• Search and Optimization: Learning to Search
• Natural Language Processing: Safety and Robustness

Application Category

• Security and Privacy: Security and privacy of machine learning and AI applications
• User Modeling, Personalization and Recommendation: User privacy protection in personalized systems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

In the era of data-driven machine-learning applications, privacy concerns and the scarcity of labeled data have become paramount challenges. These challenges are particularly pronounced in the domain of few-shot learning, where the ability to learn from limited labeled data is crucial. Privacy-preserving few-shot learning algorithms have emerged as a promising solution to address such pronounced challenges. However, it is well-known that privacy-preserving techniques often lead to a drop in utility due to the fundamental trade-off between data privacy and model performance. To enhance the utility of privacy-preserving few-shot learning methods, we introduce a novel approach called Meta-Clip. This technique is specifically designed for meta-learning algorithms, including Differentially Private (DP) model-agnostic meta-learning, DP-Reptile, and DP-MetaSGD algorithms, with the objective of balancing data privacy preservation with learning capacity maximization. By dynamically adjusting clipping thresholds during the training process, our Adaptive Clipping method provides fine-grained control over the disclosure of sensitive information, mitigating overfitting on small datasets and significantly improving the generalization performance of meta-learning models. Through comprehensive experiments on diverse benchmark datasets, we demonstrate the effectiveness of our approach in minimizing utility degradation, showcasing a superior privacy-utility trade-off compared to existing privacy-preserving techniques. The adoption of Adaptive Clipping represents a substantial step forward in the field of privacy-preserving few-shot learning, empowering the development of secure and accurate models for real-world applications, especially in scenarios where there are limited data availability.

Problem

• Balancing privacy and performance in few-shot learning
• Reducing utility drop in privacy-preserving meta-learning
• Improving generalization with adaptive clipping for small datasets

Innovation

• Dynamic clipping thresholds for privacy control
• Meta-Clip enhances DP meta-learning algorithms
• Improves generalization in few-shot learning