Spontaneous symmetry breaking and Goldstone modes for deep information propagation

2026-05-14

AI Summary

This work proposes an equivariant deep network grounded in continuous symmetry principles, introducing for the first time the Goldstone mode mechanism from physics into deep learning. By leveraging spontaneous symmetry breaking to excite Goldstone modes, the architecture enables coherent information propagation across depth and through recurrent iterations without relying on conventional stabilizing components such as residual connections or normalization layers. This approach substantially enhances trainability and representational diversity in feedforward networks and significantly improves long-sequence modeling capabilities in recurrent architectures like RNNs and GRUs. The method addresses the pervasive issue of information degradation during inter-layer or temporal propagation, offering a principled alternative to empirical stabilization techniques commonly employed in modern deep networks.

Abstract

In physical systems, whenever a continuous symmetry is spontaneously broken, the system possesses excitations called Goldstone modes, which allow coherent information propagation over long distances and times. In this work, we study deep neural networks whose internal layers are equivariant under a continuous symmetry and may therefore support analogous Goldstone-like degrees of freedom. We demonstrate, both analytically and empirically, that these degrees of freedom enable coherent signal propagation across depth and recurrent iterations, providing a mechanism for stable information flow without relying on architectural stabilizers such as residual connections or normalization. In feedforward networks, this results in improved trainability and representational diversity across layers. In recurrent settings, we demonstrate the same mechanism is valuable for long-term memory by propagating information over recurrent iterations, thereby improving performance of RNNs and GRUs on long-sequence modeling tasks.

Problem

• spontaneous symmetry breaking
• Goldstone modes
• deep information propagation
• recurrent neural networks
• signal propagation

Innovation

• spontaneous symmetry breaking
• Goldstone modes
• equivariant neural networks
• deep information propagation
• long-term memory