Memory Uncertainty Relation and Harmonic Memory in Random Recurrent Networks

2026-05-23

AI Summary

This study investigates the theoretical lower bound of short-term memory capacity in dynamical systems and its uncertainty relation with state fluctuations. By integrating analytical derivations and numerical simulations in stochastic recurrent networks subject to input noise, the work establishes—for the first time—an uncertainty inequality linking memory capacity and state variability. It further uncovers a counterintuitive phenomenon wherein noise, under regularization, can enhance memory performance. The authors introduce a suboptimal yet implementable form of memory—termed harmonic memory—realizable through readout weights, and elucidate how regularization shapes the fundamental limits of memory. The universality of the derived inequality is validated across diverse reservoir systems, while also revealing that at specific noise intensities, the original uncertainty relation breaks down, thereby advancing the understanding of the intrinsic nature of memory in dynamical systems.

Abstract

We present an inequality that bounds the short-term memory capability of dynamical systems from below. It can be interpreted as an uncertainty relation between a measure of short-term memory and that of the size of state fluctuations induced by input signals. The lower bound can be achieved by a readout weight and thus represents a suboptimal memory called harmonic memory. We examine analytically and numerically the inequality in a number of reservoir systems subject to input noise. We illustrate cases in which equality is achieved exactly, equality holds asymptotically, and the inequality is strict. We also study the effect of a state-space regularization to elucidate the inequality in terms of the fluctuation structure of the state-space. We find that a certain strength of input noise induces extra memory under the regularization, and we refer to this phenomenon as noise-induced memory. We observe that the memory uncertainty relation does not hold in general for the regularized memory and harmonic memory. This fact is explained in terms of the mechanism of noise-induced memory.

Problem

• memory uncertainty relation
• short-term memory
• random recurrent networks
• noise-induced memory
• harmonic memory

Innovation

• memory uncertainty relation
• harmonic memory
• noise-induced memory
• reservoir computing
• state-space regularization