Geometric Reciprocity: Unlocking Self-Supervision for Stereoscopic Video Generation

2026-07-06

AI Summary

This work addresses the challenge of occlusion inpainting in monocular-to-stereoscopic video conversion, a task hindered by the limited availability of real stereo data and domain shifts inherent in synthetic datasets, which constrain model generalization. The paper introduces the first self-supervised framework based on cycle consistency that learns to generate high-quality stereoscopic videos from ordinary monocular videos without requiring ground-truth stereo pairs. Its key innovation lies in incorporating the Geometric Reciprocity Theorem (GRT), which enables, for the first time, the analytical computation of occlusion masks at test time directly from monocular inputs, thereby ensuring consistency between training and inference under a nearest-neighbor depth-image-based rendering (DIBR) setup. Experiments demonstrate that the proposed method significantly outperforms both unsupervised and supervised baselines under fully unsupervised conditions, offering an effective pathway to leverage the vast abundance of monocular video content for stereoscopic generation.

Abstract

Monocular-to-stereo conversion synthesizes stereoscopic content from 2D videos for immersive 3D experiences. In modern Depth-Image-Based Rendering (DIBR) approaches, stereo inpainting of disocclusions is the critical bottleneck. Training-based methods achieve superior quality but rely on scarce stereo pairs or synthetic data with domain gaps. We address this through the first self-supervised framework learning from monocular videos via cycle consistency. Our key contribution is the Geometric Reciprocity Theorem (GRT): under the nearest-neighbor DIBR formulation, the disocclusion mask when synthesizing a target view equals the mask of pixels lost when warping back from target to source, enabling analytical computation of test-time disocclusion masks directly from monocular images. This yields train-test consistency for the stated warping formulation, supporting self-supervised learning from unlimited monocular videos and substantial improvements over training-free and supervised state-of-the-art methods. Project page: https://visual-ai.github.io/grt/

Problem

• stereoscopic video generation
• disocclusion inpainting
• monocular-to-stereo conversion
• Depth-Image-Based Rendering
• self-supervised learning

Innovation

• Geometric Reciprocity
• self-supervised learning
• stereoscopic video generation
• disocclusion inpainting
• Depth-Image-Based Rendering