Neural Computation Without Slots: Steps Towards Biologically Plausible Memory and Attention in Natural and Artificial Intelligence

2025-11-06

AI Summary

Biological brains likely lack discrete “slots” (dedicated memory units), yet mainstream AI and cognitive models often rely on slot-based assumptions for multi-item pattern storage, compromising biological plausibility. Method: We propose the K-winner Modern Hopfield Network (Kw-MHN), which replaces single-neuron memory encoding with sparse, distributed neuronal ensembles. Memory is stored via weight overlapping and retrieved through backward-time error propagation—eliminating explicit slots while emulating brain-like memory and attention. Contribution/Results: Kw-MHN significantly improves retention of old memories (superior d′ performance over baselines), enables continual learning and long-sequence processing, and successfully replicates key computational functions of slot mechanisms in language models—e.g., binding, retrieval, and compositional generalization. By unifying high capacity, biological interpretability, and scalability, Kw-MHN establishes a novel paradigm for neuroscientifically grounded memory architectures.

Technology Category

• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Machine Learning: Mixture of Experts (MoE)
• Natural Language Processing: Learning & Optimization for NLP

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Large language models for search
• Economics, Online Markets and Human Computation: Social networks and social learning

Abstract

Many models used in artificial intelligence and cognitive science rely on multi-element patterns stored in"slots"- dedicated storage locations - in a digital computer. As biological brains likely lack slots, we consider how they might achieve similar functional outcomes without them by building on the neurally-inspired modern Hopfield network (MHN; Krotov&Hopfield, 2021), which stores patterns in the connection weights of an individual neuron. We propose extensions of this approach to increase its biological plausibility as a model of memory and to capture an important advantage of slot-based computation in contemporary language models. For memory, neuroscience research suggests that the weights of overlapping sparse ensembles of neurons, rather than a dedicated individual neuron, are used to store a memory. We introduce the K-winner MHN, extending the approach to ensembles, and find that within a continual learning regime, the ensemble-based MHN exhibits greater retention of older memories, as measured by the graded sensitivity measure d', than a standard (one-neuron) MHN. Next, we consider the powerful use of slot-based memory in contemporary language models. These models use slots to store long sequences of past inputs and their learned encodings, supporting later predictions and allowing error signals to be transported backward in time to adjust weights underlying the learned encodings of these past inputs. Inspired by these models'successes, we show how the MHN can be extended to capture both of these important functional outcomes. Collectively, our modeling approaches constitute steps towards understanding how biologically plausible mechanisms can support computations that have enabled AI systems to capture human-like abilities that no prior models have been able to achieve.

Problem

• Developing biologically plausible memory models without slot-based computation
• Extending Hopfield networks to use neural ensembles for memory storage
• Enabling temporal sequence processing in biologically inspired neural networks

Innovation

• Extends Hopfield networks to neuron ensembles
• Introduces K-winner mechanism for memory retention
• Enables temporal error propagation without slots