Do Neural Networks Lose Plasticity in a Gradually Changing World?

2026-02-09

AI Summary

This work addresses the loss of plasticity in neural networks during continual learning, which is commonly attributed to architectural limitations but is here shown to stem from abrupt environmental shifts. The authors propose a framework that employs gradually evolving environments through input/output interpolation and task sampling strategies, supported by both theoretical analysis and empirical evaluation. Their findings reveal that plasticity loss is not an inherent deficiency of the model but rather an artifact induced by sudden task transitions; under gradual environmental changes, networks effectively retain their learning capacity, substantially mitigating this issue. This insight challenges a foundational assumption in current continual learning paradigms and offers a novel perspective for designing more robust and adaptive continual learning algorithms.

Technology Category

• Machine Learning: Life-Long and Continual Learning
• Search and Optimization: Learning to Search
• Natural Language Processing: Learning & Optimization for NLP

Application Category

• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Web Mining and Content Analysis: Models for Web evolution

Abstract

Continual learning has become a trending topic in machine learning. Recent studies have discovered an interesting phenomenon called loss of plasticity, referring to neural networks gradually losing the ability to learn new tasks. However, existing plasticity research largely relies on contrived settings with abrupt task transitions, which often do not reflect real-world environments. In this paper, we propose to investigate a gradually changing environment, and we simulate this by input/output interpolation and task sampling. We perform theoretical and empirical analysis, showing that the loss of plasticity is an artifact of abrupt tasks changes in the environment and can be largely mitigated if the world changes gradually.

Problem

• continual learning
• loss of plasticity
• gradual task change
• neural networks
• environmental dynamics

Innovation

• continual learning
• loss of plasticity
• gradual environment change
• task interpolation
• neural plasticity