Closed-form predictive coding via hierarchical Gaussian filters

2026-05-19

AI Summary

This work addresses the limitations of predictive coding in deep networks, which suffers from slow training and performance degradation with depth due to the neglect of precision-weighted prediction errors. The authors reformulate deep predictive coding as a hierarchical Gaussian filtering process, restoring the precision-weighted message passing required by variational inference. This enables, for the first time, closed-form variational inference in deep predictive coding without iterative relaxation or global error signals. The resulting framework supports joint online learning of activations, weights, and precisions, naturally incorporating dynamic uncertainty estimation and Hebbian-like learning properties. Experiments demonstrate that, on FashionMNIST, the method achieves training times per epoch comparable to backpropagation while converging faster, and it exhibits superior performance in online learning, data efficiency, and handling concept drift.

Abstract

Predictive coding (PC) offers a local and biologically grounded alternative to backpropagation in the training of artificial neural networks, yet to date, it remains slower, and performance degrades sharply as network depth increases. We trace both problems to a single simplification: current PC networks fix the precision matrix to the identity, discarding precision-weighted prediction errors that the variational derivation requires to be fast, local, and Bayesian. We close this gap by expressing predictive coding networks as deep hierarchical Gaussian filters (HGFs) and restore precision-weighted message passing, yielding dynamic uncertainty estimates and Hebbian-compatible update rules at every layer. The resulting networks can simultaneously learn activations, weights, and precisions under a single free-energy objective, with no global error signal, and resolve inference without requiring iterations or automatic differentiation. On FashionMNIST, our solution approaches backpropagation in epoch-level wall-clock cost while converging in fewer epochs, and outperforms it on online, data efficiency, and concept-drift tasks. We thus establish that closed-form variational inference with online precision learning provides a tractable foundation for deep predictive coding networks, retaining biological and interpretative advantages, without requiring iterative relaxation or global error signals.

Problem

• predictive coding
• precision matrix
• deep neural networks
• variational inference
• hierarchical Gaussian filters

Innovation

• predictive coding
• hierarchical Gaussian filters
• precision-weighted inference
• closed-form learning
• online uncertainty estimation