Benchmarking Predictive Coding Networks - Made Simple

2024-07-01

arXiv.org

AI Summary

To address efficiency and scalability bottlenecks in Predictive Coding Networks (PCNs), this paper introduces PCX—a lightweight, open-source library that establishes the first unified, reproducible, and extensible benchmarking framework specifically for PCNs. Methodologically, PCX provides native PyTorch implementations of core PCN algorithms and biologically inspired variants (e.g., Feedback Alignment, Direct Feedback Alignment), integrates standard computer vision and natural language processing tasks, and implements a distributed, memory-efficient training infrastructure. Key contributions include: (1) the first systematic characterization of fundamental scalability limits of PCNs with respect to parameter count and data complexity; (2) support for training PCNs with up to one million parameters—significantly exceeding prior art; and (3) new state-of-the-art performance across all benchmark tasks. The complete codebase, training scripts, and benchmark datasets are publicly released.

Technology Category

• Machine Learning: Probabilistic Circuits and Graphical Models
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Computer Vision: Learning & Optimization for CV

Application Category

• Web Mining and Content Analysis: Large pretrained models with web data
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines

Abstract

In this work, we tackle the problems of efficiency and scalability for predictive coding networks (PCNs) in machine learning. To do so, we propose a library, called PCX, that focuses on performance and simplicity, and use it to implement a large set of standard benchmarks for the community to use for their experiments. As most works in the field propose their own tasks and architectures, do not compare one against each other, and focus on small-scale tasks, a simple and fast open-source library and a comprehensive set of benchmarks would address all these concerns. Then, we perform extensive tests on such benchmarks using both existing algorithms for PCNs, as well as adaptations of other methods popular in the bio-plausible deep learning community. All this has allowed us to (i) test architectures much larger than commonly used in the literature, on more complex datasets; (ii)~reach new state-of-the-art results in all of the tasks and datasets provided; (iii)~clearly highlight what the current limitations of PCNs are, allowing us to state important future research directions. With the hope of galvanizing community efforts towards one of the main open problems in the field, scalability, we release code, tests, and benchmarks. Link to the library: https://github.com/liukidar/pcx

Problem

• Improve efficiency and scalability of PCNs
• Provide a simple, fast open-source library for benchmarks
• Identify limitations and future research directions for PCNs

Innovation

• Develops PCX library for PCNs
• Implements comprehensive benchmark tests
• Enhances scalability and performance of PCNs