Observable- and Positional-Encoding-Dependent Symmetry Readout from Neural Network Weights

2026-07-03

AI Summary

This work proposes a precise hierarchical theory of observable symmetries in neural fields, demonstrating that the symmetries apparent in network weights do not correspond to true symmetry groups but instead form an observable symmetry set jointly determined by positional encoding, training parameters, and the choice of observables. The study reveals that positional encoding not only influences function approximation capacity but also structurally constrains the extractability of symmetry information from the weights. Through experiments on two-dimensional signed distance functions using MLPs equipped with distinct positional encodings—DyadicAxisPE, TriAxisPE, and random Fourier features—the authors analyze Gram matrix–based observables and show that DyadicAxisPE supports D₄ symmetry while suppressing D₃ rotations, TriAxisPE diminishes the readout of D₃/D₆ symmetries, and random Fourier features primarily respond to π-rotational symmetry.

Abstract

Post-hoc analysis of trained neural network weights often seeks to recover geometric structure directly from the parameters. We show that, for positional-encoding-equipped neural fields, the symmetry visible from weights is not the true symmetry group itself, but an observable symmetry set determined by the trained parameters, the positional encoding (PE), and readout observable. We formulate this dependence through an exact observability hierarchy, $G_{\mathrm{obs}}^{\mathrm{exact}} \subseteq G_{\mathrm{lift}}^{\mathrm{exact}}(φ) \cap G_{\mathrm{true}}$, where $G_{\mathrm{lift}}^{\mathrm{exact}}(φ)$ is the set of input transformations that the PE can exactly lift to the feature space. The hierarchy implies that even when a target function has a geometric symmetry, that symmetry may be structurally invisible to weight-level observables if the PE does not represent the corresponding transformation. We test this prediction using MLPs trained on two-dimensional signed distance functions with multiple shape symmetry groups, positional encodings, and Gram-based observables. The results show a consistent PE-dependent pattern: DyadicAxisPE supports $D_4$-sensitive readout but structurally suppresses $D_3$ rotations, TriAxisPE yields lower $D_3$ / $D_6$ readout scores under the tested Gram observables by replacing coordinate axes with three 120-degree-separated axes, and random Fourier features mainly exhibit a $π$-rotation response under these readouts. These findings show that PE design affects not only approximation behavior but also which structures are accessible to post-hoc weight-level readouts. This provides a basis for a principled observable-dependent symmetry readout.

Problem

• symmetry readout
• positional encoding
• neural network weights
• observable dependence
• geometric symmetry

Innovation

• symmetry readout
• positional encoding
• neural fields
• observability hierarchy
• geometric structure