🤖 AI Summary
This work addresses the challenge of exactly implementing backpropagation within a physically realizable continuous-time dynamical system in finite time, circumventing the conventional reliance of energy-based models on symmetric weights or asymptotic convergence. By modeling feedforward inference as a continuous process, the authors introduce a non-conservative Lagrangian framework and construct a two-state energy functional encompassing both activations and sensitivities. The saddle-point dynamics of this functional enable simultaneous inference and credit assignment. Crucially, the study provides the first rigorous proof that standard backpropagation can be precisely replicated by a physical relaxation process in at most 2L steps for an L-layer network—without requiring weight symmetry, infinitesimal perturbations, or asymptotic assumptions—thereby enabling exact, finite-time gradient computation and offering a theoretical foundation for brain-inspired and analog hardware implementations.
📝 Abstract
Backpropagation, the foundational algorithm for training neural networks, is typically understood as a symbolic computation that recursively applies the chain rule. We show it emerges exactly as the finite-time relaxation of a physical dynamical system. By formulating feedforward inference as a continuous-time process and applying Lagrangian theory of non-conservative systems to handle asymmetric interactions, we derive a global energy functional on a doubled state space encoding both activations and sensitivities. The saddle-point dynamics of this energy perform inference and credit assignment simultaneously through local interactions. We term this framework''Dyadic Backpropagation''. Crucially, we prove that unit-step Euler discretization, the natural timescale of layer transitions, recovers standard backpropagation exactly in precisely 2L steps for an L-layer network, with no approximations. Unlike prior energy-based methods requiring symmetric weights, asymptotic convergence, or vanishing perturbations, our framework guarantees exact gradients in finite time. This establishes backpropagation as the digitally optimized shadow of a continuous physical relaxation, providing a rigorous foundation for exact gradient computation in analog and neuromorphic substrates where continuous dynamics are native.