Attribution-Based Neuron Utility for Plasticity Restoration in Deep Networks
Deep neural networks often suffer from loss of plasticity in continual learning due to neuron saturation and unbounded growth of parameter norms, hindering effective acquisition of new tasks. This work proposes the Gradient versus Reference State Discrepancy (GXD) method, which formulates adaptive resetting as an intervention cost estimation problem for the first time. By leveraging reference-based gradient attribution and a first-order Taylor expansion, GXD precisely quantifies the functional cost of resetting individual neurons, enabling identification of inefficient units and guiding adaptive reinitialization. Experimental results demonstrate that GXD significantly outperforms existing activation- or gradient-based proxy methods across diverse continual learning scenarios, effectively restoring model plasticity and learning capacity.