ICDepth: Taming Video Diffusion Models for Video Depth Estimation via In-Context Conditioning

2026-07-02

AI Summary

Monocular video depth estimation struggles to simultaneously achieve temporal consistency, geometric accuracy, and cross-scene generalization. This work proposes a novel approach based on a pretrained text-to-video diffusion Transformer, leveraging an In-Context Conditioning (ICC) mechanism to inject rich spatiotemporal priors. To ensure precise spatiotemporal alignment and suppress noise, the method introduces SAND-Attention and incorporates DINOv2-derived semantic and resolution priors through a Semantic-Resolution Fusion Module (SRFM) to enhance geometric fidelity. By integrating RoPE positional encoding and unidirectional attention, the model achieves state-of-the-art performance across multiple benchmarks using only 0.8M training frames—six to thirteen times fewer than existing methods—and demonstrates exceptional zero-shot cross-domain generalization capabilities.

Abstract

Monocular video depth estimation requires temporal consistency, geometric accuracy, and generalization across diverse scenarios, yet existing methods struggle to achieve all three simultaneously. Discriminative models excel at per-frame accuracy but suffer from temporal drift due to limited context windows, while generative methods improve consistency and generalization at the cost of extensive training data (10M+ samples) and lack of geometric precision. In response to these issues, we introduce \textbf{ICDepth}, a framework that adapts pre-trained text-to-video diffusion transformers for video depth estimation via In-Context Conditioning (ICC), leveraging their rich spatial-temporal priors. To address key challenges in transferring ICC from generation to dense prediction, we propose: (1)~\textbf{SAND-Attention}, which ensures precise spatial-temporal alignment via shared RoPE and enforces unidirectional attention to prevent noise contamination; (2)~\textbf{SRFM}, which injects DINOv2 semantic and resolution priors to enhance geometric precision. ICDepth achieves state-of-the-art results on multiple benchmarks with remarkable data efficiency, trained on only 0.8M frames ($6$--$13\times$ less than competing generative methods), while demonstrating strong zero-shot generalization to diverse domains.

Problem

• monocular video depth estimation
• temporal consistency
• geometric accuracy
• generalization
• diffusion models

Innovation

• In-Context Conditioning
• SAND-Attention
• SRFM
• Video Depth Estimation
• Diffusion Transformer