OrbitForge: Text-to-3D Scene Generation via Reconstruction-Anchored Video Synthesis

2026-06-23

AI Summary

Existing general-purpose text-to-video models struggle to generate reliable 3D assets with full viewpoint coverage and temporal consistency, primarily due to uncontrolled camera motion and missing viewpoints. This work proposes a closed-loop approach that requires neither task-specific fine-tuning nor frame-by-frame generation. By freezing a pretrained text-to-video model as a prior and integrating it with deformable Gaussian splatting for optimization-based reconstruction, the method leverages orbital rendering and missing-view detection to guide the completion of a full, closed trajectory of views. Anchoring the process in 3D reconstruction enhances three-dimensional consistency, achieving a median viewpoint span of 359.0 degrees on T3Bench and improving ImageReward from 8.07 to 16.36, with coverage quality comparable to VideoMV.

Abstract

Generic text-to-video models can be used as rich open-world scene priors. Despite the high quality of today's generated videos, they do not directly yield reliable 3D assets: camera motion is difficult to control, view coverage is partial, and frames often contain inconsistencies across time. We introduce OrbitForge, an adapter built from frozen video priors and per-prompt Gaussian Splatting reconstruction optimization that converts a single text-generated video into a canonical closed-orbit 3D Gaussian Splatting scene. We use 3D reconstruction as an anchor to improve the 3D consistency of the generated video. We obtain a preliminary 3D reconstruction from a first generated video via Deformable Gaussian Splatting with a robust MedianGS proxy. We render views from a prescribed orbit to detect missing viewpoints. OrbitForge uses the text-to-video model to complete only the missing views, and reconstructs the completed orbit into a final Gaussian Splatting scene. This design requires no task-specific video or multiview fine-tuning, avoids per-prompt score-distillation optimization, and does not progressively generate views one step at a time. We further argue that this setting demands coverage-aware evaluation: local smoothness alone rewards methods that never attempt a full orbit. On a frozen 300-prompt T3Bench-derived audit, OrbitForge reconstruction attains a 359.0-degree measured median span, raises originally unsupported-bin Q10 ImageReward from 8.07 to 16.36 relative to MedianGS-only reconstruction, while remaining competitive with VideoMV on the coverage-quality.

Problem

• text-to-3D
• 3D scene generation
• video synthesis
• view consistency
• Gaussian Splatting

Innovation

• Gaussian Splatting
• text-to-3D
• video synthesis
• 3D reconstruction
• view completion