3D Consistency Optimization for Self-Supervised Monocular Video Depth Estimation

2026-06-14

AI Summary

This work addresses the geometric inconsistency and drift commonly observed in existing self-supervised monocular video depth estimation methods, which stem from a lack of global 3D awareness. The authors reformulate the task as an unconstrained multi-view 3D reconstruction problem and introduce, for the first time, geometric priors from a 3D foundation model to establish a unified framework enforcing 3D consistency. This framework jointly optimizes three complementary constraints: image-level photometric rendering, explicit world-coordinate geometric alignment, and multi-scale temporal gradient consistency, thereby enhancing both inter-frame coherence and spatial accuracy. Experiments demonstrate that the proposed method significantly outperforms current depth estimation and 3D reconstruction baselines under both self-supervised training and zero-shot clinical scenarios.

Abstract

Reliable monocular video depth estimation is crucial for downstream 3D reasoning and embodied AI in endoscopic navigation. However, existing self-supervised approaches typically treat video frames independently or rely on weak temporal regularization. These methods, lacking a holistic perception of the underlying 3D scene, inevitably suffer from geometrically inconsistent predictions and severe cross-frame drift. To address these limitations, we introduce a new paradigm that recasts sequential video depth estimation as an unconstrained multi-view 3D reconstruction problem, enabling full exploitation of the powerful geometric priors embedded in recent 3D foundation models. The core of our approach is a 3D consistency optimization framework driven by three constraints: image-level photometric rendering, explicit world-coordinate geometric alignment, and multi-scale temporal gradient consistency. Such unified optimization elegantly anchors isolated frames to a globally coherent 3D structure. Our method has been validated in both the self-supervised training scenarios and challenging zero-shot clinical environments. Results show that the proposed approach achieves state-of-the-art spatial accuracy, outperforming the frame-based, video-based depth estimators and the multi-view 3D reconstruction baselines.

Problem

• monocular depth estimation
• 3D consistency
• self-supervised learning
• temporal drift
• geometric inconsistency

Innovation

• 3D consistency optimization
• self-supervised depth estimation
• monocular video
• multi-view 3D reconstruction
• geometric priors