Physics Question Scene Graph: Fine-grained Evaluation of Physical Plausibility in Text-to-Video Generation

2026-06-23

AI Summary

Current text-to-video generation models often violate fundamental physical laws and lack fine-grained, localized evaluation metrics. This work proposes a physics plausibility assessment framework based on a hierarchical Problem Question Semantic Graph (PQSG), which, for the first time, employs graph structures to model logical dependencies among physical reasoning questions. By integrating vision-language models with high-quality contextual examples, the framework enables precise localization and evaluation of object properties, actions, and physical consistency in generated videos using a newly curated, human-annotated benchmark dataset, FinePhyEval. Experimental results demonstrate that PQSG achieves significantly higher correlation with human judgments than existing methods, offering more accurate assessment of physical realism in state-of-the-art models such as Sora 2 and Veo 3, while also exposing critical limitations of current vision-language models in physical reasoning tasks.

Abstract

Video generation models are increasingly capable of producing realistic videos, but they still struggle to generate videos that follow basic physical laws. Compounding this is a lack of reliable granular evaluation methods for localizing and specifying physical law violations in videos. We address this by introducing Physics Question Scene Graph (PQSG), a hierarchical question-based evaluation pipeline. PQSG evaluates generated videos by checking their faithfulness to a prompt across objects, actions, and adherence to physical laws using a graph-based hierarchy of questions generated by a vision-language model (VLM), guided by high-quality in-context examples. By representing questions as a graph, PQSG introduces logical dependencies within questions, ensuring that each query is contextually valid. Moreover, PQSG provides granular assessments of which qualities of the video violate physical plausibility constraints. We validate PQSG by creating FinePhyEval, a dataset with physics-based prompts and corresponding generated videos from diverse state-of-the-art video generation models (Sora 2, Veo 3, and Wan 2.1), with each video annotated across multiple categories by humans. Using FinePhyEval, we measure the correlation between PQSG's fine-grained scores and human judgments, showing higher overall correlations than prior work. We also find that PQSG ranks closed-source models higher than Wan 2.1 on physical realism. Lastly, we show that the annotations we provide in FinePhyEval can also be used for subtask evaluation: we benchmark two strong VLMs on generating and answering questions, finding that while models can create human-like questions, they still fall short of human performance in answering them.

Problem

• physical plausibility
• text-to-video generation
• video evaluation
• physics violation
• fine-grained assessment

Innovation

• Physics Question Scene Graph
• fine-grained evaluation
• physical plausibility
• vision-language model
• video generation