Hyperspherical Forward-Forward with Prototypical Representations

2026-04-30

AI Summary

This work addresses the inefficiency of the original Forward-Forward algorithm, which requires a separate forward pass per class during inference. To overcome this limitation, the authors propose a multi-class inference method based on a hyperspherical feature space. By reformulating each layer’s local objective as a multi-class classification task on the hypersphere and learning unit-norm class prototypes as geometric anchors, the approach enables both inference and weight updates to be completed in a single forward pass while preserving the benefits of local greedy training. As the first effort to extend the Forward-Forward framework to efficient multi-class inference, the method achieves over 25% top-1 accuracy on ImageNet-1k, accelerates inference by more than 40×, and attains a transfer learning performance of 65.96%, substantially narrowing the gap with backpropagation-based methods.

Abstract

The Forward-Forward (FF) algorithm presents a compelling, bio-inspired alternative to backpropagation. However, while efficient in training, it has a computationally prohibitive inference process that requires a separate forward pass for every class that is evaluated. In this work, we introduce the Hyperspherical Forward-Forward (HFF), a novel reformulation that resolves this critical bottleneck. Our core innovation is to reframe the local objective of each layer from a binary goodness-of-fit task to a direct multi-class classification problem within a hyperspherical feature space. We achieve this by learning a set of class-specific, unit-norm prototypes that act as geometric anchors and implicit negatives. This architectural innovation preserves the benefits of local training while enabling weight update and inference in a single forward pass, making it >40x faster than the original FF algorithm. Our method is simple to implement, scales effectively to modern convolutional architectures, and achieves superior accuracy on standard image classification benchmarks, closing the gap with backpropagation. Most notably, we are among the first greedy local-learning methods to report over 25% top-1 accuracy on ImageNet-1k, and 65.96% with transfer learning.

Problem

• Forward-Forward
• inference efficiency
• local learning
• hyperspherical representation
• prototypical representations

Innovation

• Hyperspherical Forward-Forward
• prototypical representations
• local learning
• single-pass inference
• bio-inspired learning