Stimulus symmetries can confound representational similarity analyses

2026-05-20

AI Summary

This study demonstrates that stimulus symmetry can induce structurally distinct representational similarity matrices (RSMs) from functionally equivalent neural representations, thereby confounding the comparison and interpretation of neural codes. The work systematically uncovers this confounding effect for the first time, challenging the prevailing assumption in representational similarity analysis (RSA) that functionally equivalent representations are related solely by rotation. By integrating RSA with stochastic gradient descent training and energy regularization in image-encoding neural networks, the authors validate their theoretical insights. Their results reveal that symmetry gives rise to sparse and drifting neural codes, leading to unstable RSMs—a phenomenon empirically observed when networks are trained on real-world image data containing latent symmetries.

Abstract

What can representational similarity matrices (RSMs) tell us about a neural code? As the popularity of these summary statistics grows, so too does the need for a more complete characterization of their properties. Here, we show that symmetries in network inputs can confound RSM-based analyses. Stimulus symmetries render many representations functionally equivalent, but these different configurations can lead to different RSMs. These different RSMs reflect qualitatively different representational geometries. We show that stochastic gradient descent or energetic regularization can generate sparse, drifting codes, leading in turn to drifting RSMs. Moreover, we demonstrate that these phenomena are present in networks trained to encode image data, where the symmetry is latent. Our results illustrate the challenges inherent in comparing nonlinear neural codes, when functionally-equivalent representations are not related by a simple rotation.

Problem

• stimulus symmetries
• representational similarity analysis
• neural codes
• representational geometry
• functional equivalence

Innovation

• representational similarity analysis
• stimulus symmetry
• neural code
• representation geometry
• stochastic gradient descent