LikePhys: Evaluating Intuitive Physics Understanding in Video Diffusion Models via Likelihood Preference

2025-10-13

AI Summary

This work addresses the challenge of evaluating the physical understanding capability of video diffusion models. We propose LikePhys, a training-free method that leverages the denoising objective as a surrogate for the ELBO likelihood to decouple physical correctness from visual fidelity. We construct a benchmark dataset comprising 12 physically diverse scenarios, each containing paired videos—physically valid versus invalid—and introduce the Plausibility Preference Error (PPE) metric, the first automatic physical plausibility evaluator achieving high agreement with human judgments (Spearman’s ρ > 0.9). Systematic evaluation across four fundamental physics domains—rigid-body dynamics, fluid motion, collisions, and gravity—demonstrates that PPE substantially outperforms existing metrics. Furthermore, we quantitatively characterize the relationships among model architecture, sampling steps, and physical reasoning capability, and reveal systematic cross-domain disparities in physical understanding performance.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Large Multimodal Models (LMMs)
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• User Modeling, Personalization and Recommendation: Metrics for user behavior and evaluating success
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Intuitive physics understanding in video diffusion models plays an essential role in building general-purpose physically plausible world simulators, yet accurately evaluating such capacity remains a challenging task due to the difficulty in disentangling physics correctness from visual appearance in generation. To the end, we introduce LikePhys, a training-free method that evaluates intuitive physics in video diffusion models by distinguishing physically valid and impossible videos using the denoising objective as an ELBO-based likelihood surrogate on a curated dataset of valid-invalid pairs. By testing on our constructed benchmark of twelve scenarios spanning over four physics domains, we show that our evaluation metric, Plausibility Preference Error (PPE), demonstrates strong alignment with human preference, outperforming state-of-the-art evaluator baselines. We then systematically benchmark intuitive physics understanding in current video diffusion models. Our study further analyses how model design and inference settings affect intuitive physics understanding and highlights domain-specific capacity variations across physical laws. Empirical results show that, despite current models struggling with complex and chaotic dynamics, there is a clear trend of improvement in physics understanding as model capacity and inference settings scale.

Problem

• Evaluating intuitive physics understanding in video diffusion models
• Distinguishing physically valid from impossible video generations
• Analyzing model capacity effects on physical plausibility

Innovation

• Training-free evaluation method using denoising objective
• ELBO-based likelihood surrogate for physics assessment
• Benchmark testing across twelve physics scenarios