Key-Value Pair-Free Continual Learner via Task-Specific Prompt-Prototype

2026-01-01

Neural Networks

AI Summary

This work addresses the limitations of conventional prompt-based continual learning methods, which rely on key-value pair structures and suffer from inter-task interference and limited scalability. To overcome these issues, the authors propose a Prompt-and-Prototype (ProP) mechanism that dispenses with key-value pairs entirely. Instead, task-specific prompts guide feature learning, while input representations are encoded as prototypes; during inference, predictions are made by dynamically binding prompts to their corresponding prototypes. The method further incorporates prompt initialization regularization to enhance training stability. Extensive experiments on multiple standard continual learning benchmarks demonstrate that ProP substantially mitigates task interference, improves model scalability, and consistently outperforms existing approaches, thereby validating its effectiveness and superiority.

Technology Category

• Machine Learning: Life-Long and Continual Learning
• Natural Language Processing: Prompt Engineering / Prompting
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Problem

• continual learning
• prompt-based methods
• key-value pairing
• task interference
• scalability

Innovation

• continual learning
• prompt-based learning
• key-value pair-free
• task-specific prompt
• prototype