Compressed Computation is (probably) not Computation in Superposition

2026-06-12

AI Summary

This study investigates whether the Compressed Computation (CC) model genuinely achieves superposition-based computation—specifically, whether it can effectively compute 100 ReLU functions using only 50 neurons. By decomposing the training objective into ReLU-specific and input-mixing components and employing spectral analysis, subspace concentration analysis, and semi-nonnegative matrix factorization (SNMF), we demonstrate that CC’s performance gains primarily stem from an unintended mixing matrix in label space induced by residual stream noise, rather than true superposition. Removing this mixing matrix eliminates the gains, and neuron directions concentrate within the top 50 principal eigensubspaces. Building on these insights, we propose an SNMF-based baseline that successfully replicates the loss dynamics, thereby revealing the underlying mechanism of CC.

Abstract

We study whether the Compressed Computation (CC) toy model (Braun et al., 2025) is an instance of computation in superposition. The CC model appears to compute 100 ReLU functions with just 50 neurons, achieving a better loss than expected from only representing 50 ReLU functions. We show that the model mixes inputs via its noisy residual stream, corresponding to an unintended mixing matrix in the labels. Splitting the training objective into the ReLU term and the mixing term, we find that performance gains scale with the magnitude of the mixing matrix and vanish when the matrix is removed. The learned neuron directions concentrate in the subspace associated with the top 50 eigenvalues of the mixing matrix, suggesting that the mixing term governs the solution. Finally, a semi-non-negative matrix factorization (SNMF) baseline derived solely from the mixing matrix reproduces the qualitative loss profile and improves on prior baselines, though it does not match the trained model. These results suggest CC is not a suitable toy model of computation in superposition.

Problem

• Compressed Computation
• computation in superposition
• ReLU functions
• mixing matrix
• toy model

Innovation

• Compressed Computation
• computation in superposition
• mixing matrix
• semi-non-negative matrix factorization
• residual stream