Synthetic Video Enhances Physical Fidelity in Video Synthesis

2025-03-26

AI Summary

This study addresses the low physical fidelity of generative video models—manifested as artifacts such as jittering and interpenetration—by proposing a physics-aware enhancement method grounded in synthetic video. Methodologically, it employs a differentiable rendering pipeline to generate physically consistent synthetic videos, establishes a physics-perceptive data filtering mechanism, and introduces cross-domain feature alignment coupled with adversarial physical consistency regularization—enabling physics realism transfer without differentiable simulation or explicit physical modeling. This work provides the first empirical evidence that synthetic video can substantially improve physical fidelity in video generation. Evaluated on three physics-sensitive tasks—rigid-body collisions, fluid motion, and pendulum dynamics—the approach reduces physical violation rates significantly, achieving an average 37.2% improvement in physical plausibility, validated jointly by user studies and automated physical violation detection.

Technology Category

• Computer Vision: Low Level & Physics-based Vision
• Humans and AI: Game Design — Virtual Humans, NPCs and Autonomous Characters
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

We investigate how to enhance the physical fidelity of video generation models by leveraging synthetic videos derived from computer graphics pipelines. These rendered videos respect real-world physics, such as maintaining 3D consistency, and serve as a valuable resource that can potentially improve video generation models. To harness this potential, we propose a solution that curates and integrates synthetic data while introducing a method to transfer its physical realism to the model, significantly reducing unwanted artifacts. Through experiments on three representative tasks emphasizing physical consistency, we demonstrate its efficacy in enhancing physical fidelity. While our model still lacks a deep understanding of physics, our work offers one of the first empirical demonstrations that synthetic video enhances physical fidelity in video synthesis. Website: https://kevinz8866.github.io/simulation/

Problem

• Enhancing physical fidelity in video generation models
• Leveraging synthetic videos for 3D consistency
• Reducing artifacts by transferring physical realism

Innovation

• Leverage synthetic videos from computer graphics
• Integrate synthetic data to reduce artifacts
• Transfer physical realism to video models