Error Broadcast and Decorrelation as a Potential Artificial and Natural Learning Mechanism

2025-04-15

AI Summary

This paper addresses the challenge of credit assignment in deep neural networks and their reliance on backpropagation for weight transmission. To this end, we propose the Error Broadcast and Decorrelation (EBD) framework, which eliminates explicit weight feedback by directly broadcasting output errors to all layers and enforcing decorrelation regularization between layer activations and errors. Methodologically, we introduce, for the first time, a loss function derived from the minimum mean square error (MMSE) estimator’s stochastic orthogonality property, which naturally yields a biologically plausible three-factor synaptic update rule. Evaluated on multiple benchmark datasets, EBD matches or surpasses state-of-the-art error broadcast methods in accuracy while offering superior training efficiency and neuroscientific plausibility. By bridging optimization principles with biological learning mechanisms, EBD establishes a novel paradigm for biologically informed deep learning.

Technology Category

• Machine Learning: Bio-inspired Learning
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Natural Language Processing: Learning & Optimization for NLP

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Economics, Online Markets and Human Computation: Social networks and social learning
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

We introduce the Error Broadcast and Decorrelation (EBD) algorithm, a novel learning framework that addresses the credit assignment problem in neural networks by directly broadcasting output error to individual layers. Leveraging the stochastic orthogonality property of the optimal minimum mean square error (MMSE) estimator, EBD defines layerwise loss functions to penalize correlations between layer activations and output errors, offering a principled approach to error broadcasting without the need for weight transport. The optimization framework naturally leads to the experimentally observed three-factor learning rule and integrates with biologically plausible frameworks to enhance performance and plausibility. Numerical experiments demonstrate that EBD achieves performance comparable to or better than known error-broadcast methods on benchmark datasets. While the scalability of EBD to very large or complex datasets remains to be further explored, our findings suggest it provides a biologically plausible, efficient, and adaptable alternative for neural network training. This approach could inform future advancements in artificial and natural learning paradigms.

Problem

• Addresses credit assignment in neural networks via error broadcasting
• Uses stochastic orthogonality to penalize activation-error correlations
• Provides biologically plausible alternative to backpropagation

Innovation

• EBD algorithm broadcasts output error directly to layers
• Uses MMSE estimator for layerwise loss functions
• Integrates biologically plausible three-factor learning