Watching Physics: the Generative Science of Matter and Motion

2026-04-18

AI Summary

This work addresses the challenge of reliably learning and recovering the physical laws governing material motion from images and videos in the absence of explicit physical equations. It introduces a novel paradigm—“Generative Science of Matter and Motion”—that integrates Simulogenics, Physiogenics, and Materiogenics through generative video models informed by high-fidelity physical simulations, experimental observations, and mechanistic supervision. The approach is validated across three distinct systems: rubber compression, beverage can crushing, and cardiac motion. Results demonstrate that, in scenarios where kinematics are visually observable, the model accurately reconstructs measurable physical quantities such as surface strain. Moreover, the study delineates the limitations of relying solely on visual plausibility to ensure physical feasibility, thereby offering a new pathway for equipping generative models with capabilities for scientific reasoning, prediction, and design.

Technology Category

• Computer Vision: Low Level & Physics-based Vision
• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Web Mining and Content Analysis: Web data generation and simulation
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications

Abstract

Can we learn the physics of matter in motion directly from images and video--and trust it? Answering this question requires integrating experiments, physics-based simulation, and data across traditionally separate disciplines. Much of this knowledge is visual and temporal rather than textual: images and videos encode structure, dynamics, and causality that equations alone cannot fully capture. Recent generative models produce compelling visual content, yet they rely on observational data and often lack physical validity. Here we show that generative video models gain scientific value when they couple visual data with experiments and high-fidelity simulations. Using deformation mechanics as a testbed, we study three systems of increasing complexity--rubber compression, can crushing, and cardiac motion--and identify regimes in which visual learning succeeds, fails, and requires mechanistic supervision. When physics manifests in visible kinematics, generative models recover measurable quantities such as surface strain; when internal state variables dominate, visual plausibility no longer ensures physical admissibility. We propose that this convergence defines a new frontier, the Generative Sciences of Matter and Motion, which unifies Simulogenics, Physiogenics, and Materiogenics. These physics-grounded foundation models can turn visual generation into a scientific instrument for inference, prediction, and design of matter in motion.

Problem

• physics learning
• generative models
• visual data
• physical validity
• matter and motion

Innovation

• Generative Science
• Physics-Grounded Foundation Models
• Visual Learning of Physics
• Mechanistic Supervision
• Deformation Mechanics