Vector Quantization Prompting for Continual Learning

2024-10-27

Neural Information Processing Systems

AI Summary

Catastrophic forgetting in continual learning and the inability of existing prompt-based methods to jointly optimize prompt selection and task loss hinder effective knowledge retention and transfer. Method: This paper proposes a vector quantization (VQ)-based differentiable discrete prompting mechanism—introducing VQ to prompt learning for the first time—to enable end-to-end differentiable optimization of discrete prompts. By freezing the pretrained backbone and jointly optimizing prompt selection and task objectives, it overcomes the non-differentiability bottleneck of conventional identity-based prompt prediction while preserving abstraction and ensuring training stability. Contribution/Results: Adopting a class-incremental paradigm, the method achieves significant improvements over state-of-the-art approaches across multiple benchmarks, effectively mitigating catastrophic forgetting and enhancing cross-task knowledge transfer and generalization capability.

Technology Category

• Search and Optimization: Learning to Search
• Natural Language Processing: Prompt Engineering / Prompting
• Computer Vision: Diffusion Models for Vision

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Continual learning requires to overcome catastrophic forgetting when training a single model on a sequence of tasks. Recent top-performing approaches are prompt-based methods that utilize a set of learnable parameters (i.e., prompts) to encode task knowledge, from which appropriate ones are selected to guide the fixed pre-trained model in generating features tailored to a certain task. However, existing methods rely on predicting prompt identities for prompt selection, where the identity prediction process cannot be optimized with task loss. This limitation leads to sub-optimal prompt selection and inadequate adaptation of pre-trained features for a specific task. Previous efforts have tried to address this by directly generating prompts from input queries instead of selecting from a set of candidates. However, these prompts are continuous, which lack sufficient abstraction for task knowledge representation, making them less effective for continual learning. To address these challenges, we propose VQ-Prompt, a prompt-based continual learning method that incorporates Vector Quantization (VQ) into end-to-end training of a set of discrete prompts. In this way, VQ-Prompt can optimize the prompt selection process with task loss and meanwhile achieve effective abstraction of task knowledge for continual learning. Extensive experiments show that VQ-Prompt outperforms state-of-the-art continual learning methods across a variety of benchmarks under the challenging class-incremental setting. The code is available at href{https://github.com/jiaolifengmi/VQ-Prompt}{this https URL}.

Problem

• Overcoming catastrophic forgetting in continual learning
• Optimizing prompt selection with task loss
• Enhancing task knowledge abstraction with discrete prompts

Innovation

• Uses Vector Quantization for discrete prompts
• Optimizes prompt selection with task loss
• Enhances task knowledge abstraction effectively