LW2G: Learning Whether to Grow for Prompt-based Continual Learning

2024-09-27

arXiv.org

AI Summary

To address severe prompt redundancy, inefficient cross-task knowledge transfer, and low prompt selection accuracy in Prompt-based Continual Learning (PCL), this paper proposes a Dynamic Prompt Growth (DPG) mechanism. DPG introduces the Hinder Forward Capability (HFC)—a novel gradient-projection-based metric for quantifying forward transfer capability—to adaptively determine whether to instantiate new prompts. It further incorporates dynamic thresholding for decision-making and orthogonal complement space gradient constraints to preserve pretrained knowledge consistency. Additionally, a prompt weight reuse strategy is designed to enhance parameter efficiency and transfer effectiveness. Evaluated across multiple continual learning benchmarks, DPG significantly improves Prompt Retrieval Accuracy and forward transfer performance, consistently outperforming state-of-the-art methods. The implementation is publicly available.

Technology Category

• Machine Learning: Life-Long and Continual Learning
• Natural Language Processing: Prompt Engineering / Prompting
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Continual Learning (CL) aims to learn in non-stationary scenarios, progressively acquiring and maintaining knowledge from sequential tasks. Recent Prompt-based Continual Learning (PCL) has achieved remarkable performance with Pre-Trained Models (PTMs). These approaches grow a prompt sets pool by adding a new set of prompts when learning each new task (emph{prompt learning}) and adopt a matching mechanism to select the correct set for each testing sample (emph{prompt retrieval}). Previous studies focus on the latter stage by improving the matching mechanism to enhance Prompt Retrieval Accuracy (PRA). To promote cross-task knowledge facilitation and form an effective and efficient prompt sets pool, we propose a plug-in module in the former stage to extbf{Learn Whether to Grow (LW2G)} based on the disparities between tasks. Specifically, a shared set of prompts is utilized when several tasks share certain commonalities, and a new set is added when there are significant differences between the new task and previous tasks. Inspired by Gradient Projection Continual Learning, our LW2G develops a metric called Hinder Forward Capability (HFC) to measure the hindrance imposed on learning new tasks by surgically modifying the original gradient onto the orthogonal complement of the old feature space. With HFC, an automated scheme Dynamic Growing Approach adaptively learns whether to grow with a dynamic threshold. Furthermore, we design a gradient-based constraint to ensure the consistency between the updating prompts and pre-trained knowledge, and a prompts weights reusing strategy to enhance forward transfer. Extensive experiments show the effectiveness of our method. The source codes are available at url{https://github.com/RAIAN08/LW2G}.

Problem

• Addresses low selection accuracy in prompt-based continual learning
• Manages prompt pool growth to avoid unbounded cost increase
• Ensures consistency between updating prompts and pre-trained knowledge

Innovation

• Leverages task disparities for prompt pool efficiency
• Uses HFC metric to measure learning hindrance
• Dynamic Growing Approach adaptively grows prompts