Benchmarking local Hebbian learning rules for memory storage and prototype extraction

2026-05-01

AI Summary

This study investigates efficient memory storage and prototype retrieval from distorted inputs in associative memory systems, with a focus on addressing challenges posed by data correlations. For the first time, it systematically evaluates the performance of seven local Hebbian learning rules within both non-modular and modular recurrent networks, employing sparse binary patterns, winner-take-all dynamics, and recurrent architectures. The results demonstrate that the additive Hebbian rule exhibits the lowest capacity, while the covariance rule offers robustness at the cost of only moderate capacity. In contrast, the Bayesian-Hebbian rule consistently outperforms all others across nearly all conditions, achieving both the highest memory capacity and the strongest ability to extract prototypes, thereby highlighting its superior efficacy and generalization potential in associative memory tasks.

Abstract

Associative memory or content-addressable memory is an important component function in computer science and information processing, and at the same time a key concept in cognitive and computational brain science. Many different neural network architectures and learning rules have been proposed to model the brain's associative memory while investigating key component functions like figure-ground segmentation, perceptual reconstruction and rivalry. A less investigated but equally important capability of associative memory is prototype extraction where the training set comprises distorted prototype instances and the task is to recall the correct generating prototype given a new distorted instance. In this paper we benchmark associative memory function of seven different Hebbian learning rules employed in non-modular and modular recurrent networks with winner-take-all dynamics operating on moderately sparse binary patterns. We measure pattern storage and weight information capacity, prototype extraction capabilities, and sensitivity to correlations in data. The original additive Hebb rule comes out with worst capacity, covariance learning proves to be robust but with moderate capacity, and the Bayesian-Hebbian learning rules show highest capacity in almost all different conditions tested.

Problem

• associative memory
• prototype extraction
• Hebbian learning
• memory storage
• recurrent networks

Innovation

• Hebbian learning
• associative memory
• prototype extraction
• Bayesian-Hebbian
• winner-take-all dynamics