What Accuracy and Gradient Cosine Miss: Evaluating Feedback Alignment via Scale Stability, Reference Validity, and Depth Utility

2026-06-19

AI Summary

This work addresses critical limitations in the evaluation of existing feedback alignment methods, which rely primarily on task accuracy and aggregated gradient cosine similarity and thus fail to uncover two silent failure modes: reference gradient degradation and inter-layer heterogeneity in credit assignment. To resolve this, the authors propose a novel diagnostic evaluation protocol that introduces three layer-wise checks—scale stability, reference validity, and depth utility—combined with fine-grained cosine analysis. This framework enables, for the first time, the explicit identification and disentanglement of the aforementioned failure mechanisms. Empirical validation across diverse architectures and algorithms demonstrates its ability to detect failure cases missed by conventional metrics and to provide actionable insights for improvement, confirming both its broad applicability and high diagnostic precision.

Abstract

Despite the success of deep learning, training deep networks in biologically plausible and hardware-efficient ways remains an open challenge. Feedback alignment (FA) methods address this by replacing backpropagation's symmetric backward weights with fixed random matrices, but their effectiveness depends critically on whether they can be accurately evaluated. The standard evaluation relies on two quantities: task accuracy and cosine similarity between the method's credit signal and the backpropagation gradient. We show that this reporting pair is insufficient by identifying two independent failure modes, both silent under current reporting: (1) measurement degeneracy, where the BP reference gradient collapses to the numerical floor in terminal-LayerNorm residual architectures, rendering cosine uninterpretable; and (2) aggregation collapse, where the aggregate cosine masks layerwise heterogeneity that concentrates credit at one end of the network. To address these limitations, we propose a diagnostic evaluation protocol based on three checks -- scale stability, reference validity, and depth utility -- together with per-layer rather than aggregate cosine reporting. Across multiple architectures and methods, the standard reporting pair gives no signal of failure in any audited case, while our protocol identifies all failures with wide calibration margins. The two failure modes are causally independent: a per-block scale penalty alleviates Mode 1 (residual scale explosion driving reference collapse) without affecting Mode 2 (cosine ranking that contradicts every functional metric we measured). Identifying these silent failures prevents researchers from building on non-functional credit assignment and provides actionable guidance for developing FA methods that genuinely train deep layers.

Problem

• Feedback Alignment
• Gradient Cosine
• Scale Stability
• Reference Validity
• Depth Utility

Innovation

• Feedback Alignment
• Gradient Cosine
• Scale Stability
• Reference Validity
• Depth Utility