NORACL: Neurogenesis for Oracle-free Resource-Adaptive Continual Learning

2026-04-29

AI Summary

This work addresses the stability-plasticity dilemma in continual learning, where fixed network capacity struggles to accommodate unknown task sequences. Inspired by biological neurogenesis, the authors propose a resource-adaptive method that operates without prior task information. By monitoring representation drift and plasticity saturation signals, the approach dynamically triggers neuron growth and directs new units to either feature extraction or composition layers based on task similarity. This study is the first to integrate neurogenesis into continual learning, eliminating reliance on pre-specified oracle architectures and yielding an interpretable, task-driven network structure. Experiments demonstrate that the method achieves or surpasses the average accuracy of oracle static baselines across diverse task settings while using fewer parameters, and further uncovers the fundamental cause of plasticity decay in fixed-capacity networks.

Abstract

In a continual learning setting, we require a model to be plastic enough to learn a new task and stable enough to not disturb previously learned capabilities. We argue that this dilemma has an architectural root. A finite network has limited representational and plastic resources, yet the required capacity depends on properties of the future task stream that are unknown: how many tasks will be encountered, and how much they overlap in feature space. Regularization-based methods preserve past knowledge within fixed-capacity architectures and therefore implicitly rely on an oracle architecture sized for this unknown future. When tasks are only weakly related, fixed architectures progressively run out of plastic resources; when tasks are few or strongly overlapping, models are often over-provisioned. Inspired by neurogenesis in biology, we propose NORACL to address the stability-plasticity dilemma by tackling the oracle architecture problem through neuronal growth. Starting from a compact network, NORACL grows only when needed by monitoring two complementary signals for representational and plasticity saturation. We evaluate NORACL against oracle-sized static baselines across varying task counts and geometries. Across all settings, NORACL achieves final average accuracies that are better than or on par with oracle-provisioned static baselines while using fewer parameters. Additionally, NORACL yields architectures with interpretable growth, i.e. dissimilar tasks predominantly expand feature-extraction layers, whereas tasks which rely on common features shift growth toward later feature-combination layers. Our analysis further explains why fixed-capacity networks lose plasticity as tasks accumulate, whereas NORACL creates fresh capacity for new tasks through growth. Together, these results show that adaptive neurogenesis pushes the stability-plasticity Pareto frontier of continual learning.

Problem

• continual learning
• stability-plasticity dilemma
• oracle architecture
• resource adaptation
• neurogenesis

Innovation

• neurogenesis
• continual learning
• resource-adaptive
• stability-plasticity dilemma
• dynamic network growth