Task-Restricted Symmetries in Recurrent Weight Space
This study investigates task-dependent functional redundancy and approximate invariance in recurrent neural networks. By applying the real Schur decomposition, the recurrent weight matrix is decoupled into spectral blocks and non-normal coupling structures. Structured ablation is then performed while preserving the input–output mapping to identify weight perturbations that do not significantly affect task performance. The work introduces the notion of “task-dependent approximate functional invariance,” revealing non-universal yet task-specific symmetries within recurrent architectures. Experiments across dynamic tasks—including copying, flip-flop triggering, sine wave generation, and context-dependent integration—demonstrate that certain non-normal Schur couplings can be safely removed, while others are essential for autonomous replay. These findings validate the existence of task-constrained symmetries and provide an interpretable diagnostic framework for analyzing recurrent network dynamics.