Fast and Slow Variational Continual Learning

2026-06-22

AI Summary

This work addresses the stability-plasticity imbalance in deep continual learning, which arises from the lack of intrinsic adaptation mechanisms in standard optimizers. Building upon the Variational Continual Learning (VCL) framework, the authors propose CoVON—an optimizer that uniquely integrates neuroscience-inspired fast and slow adaptation mechanisms within IVON. Slow adaptation is achieved by merging historical posterior distributions to mitigate catastrophic forgetting, while fast weight updates are preserved to maintain plasticity. CoVON achieves this integration without substantial computational overhead and consistently outperforms existing VCL approaches and weight regularization strategies across diverse continual learning scenarios, including domain-incremental learning, continual pre-training, and fine-tuning of large language models.

Abstract

Continual learning remains a major challenge for modern deep networks, partly because commonly used optimizers lack inherent mechanisms for continual adaptation. One such natural mechanism is fast and slow adaptation to balance stability and plasticity. This mechanism has deep roots in neuroscience and biology, but there is no consensus on how to best incorporate it in commonly used optimizers. Here, we show that this can be easily done via the VCL framework, where past posteriors are used as priors in the future. Our key idea is to incorporate slow adaptation via merging of past posteriors to slow down the drift in the knowledge as learning progresses. The merged posterior is then used as the prior in the VCL update to implement the fast-weight updates. These steps can be seamlessly implemented in the IVON optimizer, whose form and costs are nearly identical to that of Adam. We call this new optimizer the Continual IVON (CoVON) optimizer and show that it not only consistently improves over existing VCL optimizers, but also performs better than other weight-regularization strategies across domain-incremental learning, continual pre-training, and fine-tuning of large language models.

Problem

• continual learning
• stability-plasticity dilemma
• deep neural networks
• optimizers
• adaptation

Innovation

• variational continual learning
• fast and slow adaptation
• posterior merging
• IVON optimizer
• stability-plasticity trade-off