Backpropagation as Physical Relaxation: Exact Gradients in Finite Time
This work addresses the challenge of exactly implementing backpropagation within a physically realizable continuous-time dynamical system in finite time, circumventing the conventional reliance of energy-based models on symmetric weights or asymptotic convergence. By modeling feedforward inference as a continuous process, the authors introduce a non-conservative Lagrangian framework and construct a two-state energy functional encompassing both activations and sensitivities. The saddle-point dynamics of this functional enable simultaneous inference and credit assignment. Crucially, the study provides the first rigorous proof that standard backpropagation can be precisely replicated by a physical relaxation process in at most 2L steps for an L-layer network—without requiring weight symmetry, infinitesimal perturbations, or asymptotic assumptions—thereby enabling exact, finite-time gradient computation and offering a theoretical foundation for brain-inspired and analog hardware implementations.