Rank-Order N-of-M Codes for Sparse Distributed Memory: Disentangling Representation and Learning Effects in Noise Robustness Against Contemporary Neuromorphic Architectures

2026-07-03

AI Summary

This study addresses the noise robustness and capacity limitations of Sparse Distributed Memory (SDM) in continual learning, which stem from its encoding mechanism. The authors propose and evaluate a synergistic framework combining Rank-Order N-of-M coding with MAX-Hebbian learning rules. For the first time, they explicitly disentangle the individual contributions of encoding representations and learning mechanisms to robustness, demonstrating that their integration substantially enhances performance. Experiments conducted on the WheelSDM architecture using GloVe-100 embeddings and sentence embeddings show that, at saturation, the proposed approach achieves a 13.4 percentage point improvement in memory capacity over StandardSDM. Furthermore, rank-order coding consistently yields robustness gains while consuming only half the energy of SpikingMamba SI-LIF neurons.

Abstract

Large language models remain limited as continual learning systems, motivating renewed interest in Sparse Distributed Memory (SDM) as an explicit online episodic memory. CALM (Nechesov and Ruponen, 2025) identifies its threshold-binary encoder as an open design question. This paper evaluates rank-order N-of-M encoding (Furber et al., 2007) as an alternative. We make three contributions. First, a faithful reimplementation validates the published architecture by confirming exact equivalence between WheelSDM and RankOrderSDM (cosine similarity 1.0000 across 10 seeds) and reproducing the documented divergence of RDLIF neurons under interference. Second, multi-seed capacity experiments show RankOrderSDM outperforming StandardSDM by 13.4 percentage points at saturation in the scaled configuration and by 0.8 percentage points at the published architecture scale. Third, BER robustness experiments disentangle representation and learning effects, showing that the large robustness gain arises primarily from the interaction of rank-order encoding with MAX-Hebbian learning, while the encoder alone provides only a small advantage under matched learning conditions. Experiments on GloVe-100 embeddings confirm this small but consistent encoding benefit on real structured data, whereas sentence embeddings exhibit a ceiling effect at low memory load. A secondary analysis shows that idealized rank-order encoding requires half the component-level encoding energy of SpikingMamba's SI-LIF neurons at four-bit precision, although decoder costs dominate overall system energy. These results identify which components of the original rank-order SDM architecture provide measurable benefits for contemporary memory-augmented AI systems, offering practical guidance for architectures such as CALM.

Problem

• Sparse Distributed Memory
• noise robustness
• rank-order encoding
• neuromorphic architectures
• representation vs learning

Innovation

• Rank-Order N-of-M Encoding
• Sparse Distributed Memory
• MAX-Hebbian Learning
• Noise Robustness
• Energy Efficiency