The Objective Decides: When a Learned Dynamics Model Uses a Conserved Quantity

2026-07-04

AI Summary

While existing linear probes can decode conserved quantities with high accuracy, they cannot determine whether a dynamics model causally relies on them for predictions. This work introduces the concept of “causal deployment,” combining single-step activation swapping, a causal effect metric (transfer-corr τ), and algebraic predicate analysis to systematically evaluate the causal role of conserved quantities in prediction. The study reveals that whether a model deploys conserved quantities depends on its training objective and is governed by the algebraic structure of its outputs; notably, even when probe R² ≈ 1, conserved quantities may remain causally inert. For the first time, the research demonstrates a strong correlation (r = +0.97) between deployment gaps and out-of-distribution (OOD) performance, validating this finding across diverse physical systems and a 158-million-parameter PDE foundation model.

Abstract

A linear probe that recovers a conserved quantity from a learned dynamics model's activations is routinely read as evidence that the model uses that quantity. We show this inference is unsound. Across mechanical, circuit, and partial-differential-equation (PDE) systems, and on a 158M-parameter pretrained PDE foundation model, energy and other invariants are linearly decodable at $R^2 \approx 1$ yet causally inert on next-state prediction: overwriting the decoded direction with a donor state's value (single-step activation interchange) leaves the forward pass essentially unchanged (transfer-corr $τ\approx 0$). The same direction in the same representation becomes causally load-bearing ($τ\to +1$) the moment the training objective rewards the invariant, so deployment is a property of the objective, not of the representation or the probe. We further show that when an invariant is deployed is governed by a precise algebraic predicate (its relation to the prediction output), by flipping a single invariant from inert to load-bearing by changing only the output's algebra. Finally, the gap has teeth: across models that all decode the target at $R^2 = 1.00$, the deployment gap forecasts out-of-distribution (OOD) accuracy ($r = +0.97$) where decodability is blind. We argue that causal deployment, not decodability, is what interpretability should measure when the question is whether a model uses a piece of knowledge, and we give a cheap instrument for measuring it.

Problem

• conserved quantity
• learned dynamics model
• linear probe
• causal deployment
• interpretability

Innovation

• causal deployment
• linear probe
• conserved quantity
• activation interchange
• out-of-distribution generalization