GeoT2V-Bench: Benchmarking 3D Consistency in Text-to-Video Models via 3D Reconstruction

2026-06-23

AI Summary

Existing text-to-video models struggle to maintain geometric consistency in static 3D scenes when generating videos conditioned on camera prompts. To address this, this work proposes the first continuous evaluation framework for assessing 3D consistency in text-to-video generation. The framework estimates per-frame camera parameters, fits deformable Gaussian splats, and aggregates them into a static median representation to diagnose the feasibility of rigid 3D reconstruction. Integrating VGGT-style geometric estimation, camera trajectory optimization, and optical flow consistency analysis, it conducts 3,840 reconstruction trials across 12 open-source models and 80 static scene prompts. This comprehensive approach reveals multidimensional failure modes—including motion appearance artifacts, static rendering errors, and optical flow inconsistencies—thereby surpassing simplistic scoring or binary judgments and systematically uncovering complementary 3D consistency deficiencies across models.

Abstract

Camera-prompted text-to-video (T2V) models are increasingly used to synthesize virtual camera captures, such as orbiting objects or moving through static scenes. For these outputs, visual plausibility is insufficient: the generated frames should also provide coherent multi-view evidence for a single static 3D scene. We introduce GeoT2V-Bench, a reconstruction-based diagnostic benchmark for evaluating whether camera-prompted T2V clips can support explicit rigid 3D reconstruction. Our pipeline estimates per-frame camera intrinsics and poses with VGGT-style geometry estimation, fits DeformableGS, derives a static MedianGS proxy by temporal-median aggregation, and renders this proxy along the estimated camera path. Instead of producing a pass/fail label or a single scalar score, GeoT2V-Bench reports a continuous reconstruction profile covering apparent image motion, estimated trajectory behavior, MedianGS static rendering error, static-render flow agreement, and the gap between flexible and static fits. On a fair-format four-seed evaluation with 3,840 completed reconstructions from 12 open-weight model configurations and 80 GeCo-Eval static-scene prompts, we find that visible motion, static rendering error, flow agreement, and flexible-vs-static behavior often disagree. GeoT2V-Bench therefore captures complementary failure modes that emerge when generated videos are tested as global static-scene acquisitions.

Problem

• 3D consistency
• text-to-video
• 3D reconstruction
• camera-prompted generation
• static scene

Innovation

• 3D consistency
• text-to-video
• 3D reconstruction
• camera-prompted generation
• Geometric evaluation