A Survey on Brain-Inspired Deep Learning via Predictive Coding

2023-08-15

AI Summary

Deep neural networks suffer from high computational cost, limited robustness, poor biological plausibility, and inadequate uncertainty quantification. Method: This work systematically reviews predictive coding (PC)-driven brain-inspired deep learning, unifying its theoretical foundations within a variational inference framework for the first time. It integrates neuroscience principles, Bayesian generative modeling, variational inference, and continuous dynamical systems to construct the first comprehensive research map of PC-enabled machine learning. Contribution/Results: The analysis reveals PC’s cross-task generalization capabilities in multi-regional dynamic brain modeling, cognitive control, and embodied intelligence. Critically, PC reduces reliance on backpropagation-based gradient computation while natively supporting uncertainty-aware inference and online adaptation. These properties establish PC as a foundational paradigm for developing next-generation learning systems that are biologically interpretable, computationally efficient, robust, and trustworthy.

Abstract

Artificial intelligence (AI) is rapidly becoming one of the key technologies of this century. The majority of results in AI thus far have been achieved using deep neural networks trained with the error backpropagation learning algorithm. However, the ubiquitous adoption of this approach has highlighted some important limitations such as substantial computational cost, difficulty in quantifying uncertainty, lack of robustness, unreliability, and biological implausibility. It is possible that addressing these limitations may require schemes that are inspired and guided by neuroscience theories. One such theory, called predictive coding (PC), has shown promising performance in machine intelligence tasks, exhibiting exciting properties that make it potentially valuable for the machine learning community: PC can model information processing in different brain areas, can be used in cognitive control and robotics, and has a solid mathematical grounding in variational inference, offering a powerful inversion scheme for a specific class of continuous-state generative models. With the hope of foregrounding research in this direction, we survey the literature that has contributed to this perspective, highlighting the many ways that PC might play a role in the future of machine learning and computational intelligence at large.

Problem

• Deep Neural Networks
• Computational Efficiency
• Brain-inspired Learning

Innovation

• Predictive Coding
• Deep Learning
• Neural Efficiency