Self-supervised local learning rules learn the hidden hierarchical structure of high-dimensional data

2026-05-18

AI Summary

This study investigates how biologically plausible local learning rules can efficiently uncover latent hierarchical structures in high-dimensional data without relying on backpropagation’s symmetric error pathways or prolonged training. The authors evaluate two classes of rules on a random hierarchical model (RHM): one approximating output errors via direct feedback, and the other employing layer-wise self-supervised contrastive or non-contrastive losses that operate without explicit error signals. Their findings demonstrate that input-specific nonlinearities—such as masking—are critical for learning complex tasks. Notably, the second class of self-supervised rules not only effectively recovers the RHM’s hidden hierarchy with data efficiency comparable to supervised backpropagation but also aligns closely with known mechanisms of local synaptic plasticity in the cortex.

Abstract

The brain learns abstract representations of high-dimensional sensory input, but the plasticity rules that enable such learning are unknown. We study biologically plausible algorithms on the Random Hierarchy Model (RHM), an artificial dataset designed to investigate how deep neural networks learn the intrinsic hierarchical structure of high-dimensional data. We focus on two types of local learning rules that avoid both a long convergence time and the use of a symmetric error network. The first type uses direct feedback signals to approximate error propagation from the output layer. The second type uses layerwise self-supervised contrastive or non-contrastive loss functions that do not explicitly approximate errors at the output layer. We show that all rules of the first type fail to solve the tasks of the RHM and trace this failure back to input-specific nonlinearities (`masking') that are implemented in full backpropagation and are essential for learning complex tasks. However, algorithms of the second type are able to learn the hierarchical hidden structure of the RHM tasks and are as data-efficient as supervised backpropagation training, while being compatible with known rules of synaptic plasticity in cortex.

Problem

• self-supervised learning
• hierarchical structure
• local learning rules
• synaptic plasticity
• high-dimensional data

Innovation

• self-supervised learning
• local learning rules
• hierarchical representation
• synaptic plasticity
• Random Hierarchy Model