StereoWorld: Geometry-Aware Monocular-to-Stereo Video Generation

2025-12-10

AI Summary

Monocular-to-stereoscopic video generation faces challenges including high computational cost, severe visual artifacts, and inconsistent 3D geometry—particularly problematic for XR applications requiring natural inter-pupillary distance (IPD) alignment and high-fidelity stereo output. Method: We propose an end-to-end, stereo-supervision-free generative framework built upon a pre-trained video diffusion model. Our approach introduces a novel geometry-aware regularization mechanism and integrates spatio-temporal tiling synthesis to jointly leverage monocular video conditioning and implicit geometric constraints. Contribution/Results: Leveraging a large-scale, self-collected dataset of 11 million HD stereo video frames, our method achieves high-resolution stereoscopic video generation with strong geometric consistency. Quantitative and qualitative evaluations demonstrate significant improvements over state-of-the-art methods in both perceptual quality and depth consistency, validating its suitability for real-world XR deployment.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Natural Language Processing: Generation
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG

Abstract

The growing adoption of XR devices has fueled strong demand for high-quality stereo video, yet its production remains costly and artifact-prone. To address this challenge, we present StereoWorld, an end-to-end framework that repurposes a pretrained video generator for high-fidelity monocular-to-stereo video generation. Our framework jointly conditions the model on the monocular video input while explicitly supervising the generation with a geometry-aware regularization to ensure 3D structural fidelity. A spatio-temporal tiling scheme is further integrated to enable efficient, high-resolution synthesis. To enable large-scale training and evaluation, we curate a high-definition stereo video dataset containing over 11M frames aligned to natural human interpupillary distance (IPD). Extensive experiments demonstrate that StereoWorld substantially outperforms prior methods, generating stereo videos with superior visual fidelity and geometric consistency. The project webpage is available at https://ke-xing.github.io/StereoWorld/.

Problem

• Generates high-fidelity stereo video from monocular input
• Ensures 3D structural fidelity with geometry-aware regularization
• Enables efficient high-resolution synthesis via spatio-temporal tiling

Innovation

• Repurposes pretrained video generator for stereo conversion
• Uses geometry-aware regularization for 3D structural fidelity
• Integrates spatio-temporal tiling for high-resolution synthesis