Unlocked Backpropagation using Wave Scattering

2026-02-11

AI Summary

This work addresses the “forward–backward” locking inherent in classical backpropagation and optimal control, which arises from two-point boundary value problems. By introducing an “optimization time” dimension, the authors reformulate Pontryagin’s maximum principle as a hyperbolic initial value problem. Leveraging backward-propagating wave variables with finite propagation speed, the optimization dynamics are characterized through scattering relations, thereby incorporating wave scattering and dissipation mechanisms into neural network training for the first time and eliminating temporal synchronization constraints. Grounded in continuum mechanics, hyperbolic partial differential equations, and parameter-port reflection minimization, this framework yields a family of fully unlocked training algorithms and reveals that any physical system supporting wave scattering and dissipation can naturally perform optimization computations.

Technology Category

• Machine Learning: Optimization
• Search and Optimization: Learning to Search
• Natural Language Processing: Learning & Optimization for NLP

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Social Networks and Social Media: Influence propagation, information diffusion, and the prediction on networks

Abstract

Both the backpropagation algorithm in machine learning and the maximum principle in optimal control theory are posed as a two-point boundary problem, resulting in a"forward-backward"lock. We derive a reformulation of the maximum principle in optimal control theory as a hyperbolic initial value problem by introducing an additional"optimization time"dimension. We introduce counter-propagating wave variables with finite propagation speed and recast the optimization problem in terms of scattering relationships between them. This relaxation of the original problem can be interpreted as a physical system that equilibrates and changes its physical properties in order to minimize reflections. We discretize this continuum theory to derive a family of fully unlocked algorithms suitable for training neural networks. Different parameter dynamics, including gradient descent, can be derived by demanding dissipation and minimization of reflections at parameter ports. These results also imply that any physical substrate that supports the scattering and dissipation of waves can be interpreted as solving an optimization problem.

Problem

• backpropagation
• optimal control
• two-point boundary problem
• wave scattering
• unlocked algorithms

Innovation

• wave scattering
• unlocked backpropagation
• optimal control
• hyperbolic initial value problem
• physical optimization