The Observer Effect in World Models: Invasive Adaptation Corrupts Latent Physics

2026-02-12

AI Summary

Existing methods struggle to determine whether neural world models genuinely internalize physical laws or merely exploit statistical shortcuts, particularly in out-of-distribution settings. This work proposes PhyIP, a non-invasive evaluation protocol that assesses the decodability of key physical quantities—such as internal energy and inverse-square laws—using low-capacity linear probes on frozen self-supervised representations, thereby avoiding fine-tuning or other invasive operations that could disrupt latent physical structure. Experiments in fluid dynamics and orbital mechanics demonstrate that PhyIP efficiently recovers ground-truth physical quantities under out-of-distribution conditions (correlation coefficient ρ > 0.90), whereas invasive approaches cause representational collapse (ρ ≈ 0.05). This study thus reveals, for the first time, the critical influence of evaluation methodology on judgments of physical internalization.

Technology Category

• Computer Vision: Low Level & Physics-based Vision
• Machine Learning: Calibration & Uncertainty Quantification
• Natural Language Processing: Interpretability, Analysis, and Evaluation of NLP Models

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• User Modeling, Personalization and Recommendation: Metrics for user behavior and evaluating success
• Security and Privacy: Large-scale security measurements

Abstract

Determining whether neural models internalize physical laws as world models, rather than exploiting statistical shortcuts, remains challenging, especially under out-of-distribution (OOD) shifts. Standard evaluations often test latent capability via downstream adaptation (e.g., fine-tuning or high-capacity probes), but such interventions can change the representations being measured and thus confound what was learned during self-supervised learning (SSL). We propose a non-invasive evaluation protocol, PhyIP. We test whether physical quantities are linearly decodable from frozen representations, motivated by the linear representation hypothesis. Across fluid dynamics and orbital mechanics, we find that when SSL achieves low error, latent structure becomes linearly accessible. PhyIP recovers internal energy and Newtonian inverse-square scaling on OOD tests (e.g., $\rho>0.90$). In contrast, adaptation-based evaluations can collapse this structure ($\rho \approx 0.05$). These findings suggest that adaptation-based evaluation can obscure latent structures and that low-capacity probes offer a more accurate evaluation of physical world models.

Problem

• world models
• observer effect
• out-of-distribution
• self-supervised learning
• physical laws

Innovation

• non-invasive evaluation
• world models
• self-supervised learning
• linear representation hypothesis
• out-of-distribution generalization