A Study of Hidden-State Optimization Order in Predictive Coding Networks

2026-09-01

AI Summary

研究通过在预测编码网络中采用边界优先的优化顺序，解决了局部学习方法在深层网络中特征学习弱的问题，提高了CIFAR-10上的准确率。

Abstract

Local learning methods offer an alternative to end-to-end backpropagation, but their unstructured local objectives can produce weak feature learning in deep networks. We study whether the order of hidden-state optimization can address this limitation. We propose a boundary-first inference schedule that partitions a model into chunks, first coordinates hidden states at chunk boundaries, and then refines representations within each chunk. We instantiate this schedule in predictive coding networks (PCNs), a local-learning framework in which hidden activities and prediction errors are explicitly exposed during inference. On CIFAR-10, the resulting boundary-first predictive-coding instantiation improves accuracy over standard predictive coding by $9.77\%$ under a standard parametrization and by $5.51\%$ under a $μ$-parametrization. Diagnostic analyses further show more non-trivial early-layer updates, lower initial-to-final CKA, and more diverse layerwise gradients, consistent with stronger feature learning. These results support boundary-first, chunk-based inference as a practical design principle for predictive-coding training and motivate its study in broader local-learning systems.

Problem

• local learning
• hidden-state optimization
• feature learning
• predictive coding networks

Innovation

• boundary-first inference
• chunk-based optimization
• predictive coding networks
• local learning