Dendritic In-Context Learning in a Single-Layer Spiking Neural Network

2026-07-02

AI Summary

Current spiking neural networks (SNNs) exhibit limited performance in high-dimensional contextual learning tasks and lack biologically plausible implicit learning capabilities. This work proposes DendriCL, a novel architecture that, for the first time, treats dendritic compartments as fundamental computational units. By leveraging subthreshold dendritic dynamics within a single-layer SNN, DendriCL enables online learning without requiring attention mechanisms, deep architectures, or synaptic plasticity during inference. The approach is grounded in a leaky online Widrow-Hoff least mean squares (LMS) algorithm and demonstrates seed stability on the Garg-2022 hyperdimensional contextual learning benchmark. A linear probe applied to apical membrane potentials accurately reconstructs the LMS trajectory with an R² of 0.93, significantly outperforming dense Transformers and challenging conventional SNN design paradigms.

Abstract

In-context learning (ICL) operates via implicit gradient descent embedded in the forward pass of modern AI architectures -- Transformers, Mamba, state-space models, and MLPs. Capturing this capability in biologically plausible Spiking Neural Networks (SNNs) has remained an open challenge: existing SNNs fail the Garg-2022 benchmark at non-trivial task dimensions. We trace this failure to a structural assumption: prior SNN designs route adaptation through inference-time synaptic plasticity, viewing the dendritic compartment as a passive conduit for error or teacher signals. We challenge this assumption. The subthreshold dynamics of a single dendritic compartment already implement a complete online learning algorithm. By treating the compartment as the computational substrate rather than a passive conduit, we propose DendriCL -- a single-layer compartmental spiking architecture whose apical recurrence is structurally identical to leaky online Widrow-Hoff LMS. This dynamics-only update collapses the architectural depth required for general-purpose ICL to a single layer. DendriCL is uniquely seed-stable at super-dimensional Garg-2022 ICL -- where dense Transformers exhibit grokking-style instability and fail past moderate task dimension -- and a linear probe recovers the reference online-LMS trajectory directly from the apical membrane at R^2 = 0.93, showing the algorithm is structurally embedded in the dynamics rather than implicitly discovered during training. Taken together, ICL requires neither attention, depth, nor inference-time plasticity: a single compartment with online-LMS dynamics is sufficient.

Problem

• In-Context Learning
• Spiking Neural Networks
• Dendritic Computation
• Online Learning
• Biological Plausibility

Innovation

• In-context learning
• Spiking Neural Networks
• Dendritic computation
• Online LMS
• Biologically plausible AI