Parameter-Efficient Continual Fine-Tuning: A Survey

2025-04-18

AI Summary

This paper addresses the fundamental challenge of balancing catastrophic forgetting and parameter efficiency when large pre-trained models continuously adapt to dynamic task streams. To this end, we propose the first unified theoretical framework for Parameter-Efficient Continual Fine-Tuning (PECFT). Our framework systematically organizes existing approaches along three dimensions: method taxonomy, evaluation metrics, and core challenges—integrating Parameter-Efficient Fine-Tuning (PEFT) techniques (e.g., adapters, LoRA, prompt tuning) with continual learning strategies (e.g., replay, regularization, architecture expansion). Through a comprehensive review of over 100 studies, we identify key trade-offs between performance and efficiency, and pinpoint scalable memory mechanisms and task-aware parameter updates as critical research frontiers. This work bridges a significant gap at the intersection of continual learning and PEFT, providing both theoretical foundations and practical guidelines for efficient, sustainable adaptation of large language models.

Technology Category

• Machine Learning: Life-Long and Continual Learning
• Natural Language Processing: (Large) Language Models
• Cognitive Modeling & Cognitive Systems: Adaptive Behavior

Application Category

• User Modeling, Personalization and Recommendation: Practical large-scale studies of user experience
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

The emergence of large pre-trained networks has revolutionized the AI field, unlocking new possibilities and achieving unprecedented performance. However, these models inherit a fundamental limitation from traditional Machine Learning approaches: their strong dependence on the extit{i.i.d.} assumption hinders their adaptability to dynamic learning scenarios. We believe the next breakthrough in AI lies in enabling efficient adaptation to evolving environments -- such as the real world -- where new data and tasks arrive sequentially. This challenge defines the field of Continual Learning (CL), a Machine Learning paradigm focused on developing lifelong learning neural models. One alternative to efficiently adapt these large-scale models is known Parameter-Efficient Fine-Tuning (PEFT). These methods tackle the issue of adapting the model to a particular data or scenario by performing small and efficient modifications, achieving similar performance to full fine-tuning. However, these techniques still lack the ability to adjust the model to multiple tasks continually, as they suffer from the issue of Catastrophic Forgetting. In this survey, we first provide an overview of CL algorithms and PEFT methods before reviewing the state-of-the-art on Parameter-Efficient Continual Fine-Tuning (PECFT). We examine various approaches, discuss evaluation metrics, and explore potential future research directions. Our goal is to highlight the synergy between CL and Parameter-Efficient Fine-Tuning, guide researchers in this field, and pave the way for novel future research directions.

Problem

• Addressing catastrophic forgetting in continual learning scenarios
• Enhancing parameter-efficient fine-tuning for dynamic environments
• Surveying methods for lifelong adaptation of large pre-trained models

Innovation

• Parameter-Efficient Fine-Tuning for dynamic adaptation
• Combining Continual Learning with PEFT methods
• Addressing Catastrophic Forgetting in sequential tasks