StableDPT: Temporal Stable Monocular Video Depth Estimation

2026-01-06

arXiv.org

AI Summary

This work addresses the temporal inconsistency and flickering artifacts commonly observed when applying monocular depth estimation models directly to video sequences. To this end, the authors propose a lightweight, trainable temporal module that enhances temporal coherence by integrating global temporal context through cross-frame keyframe sampling and an efficient cross-attention mechanism. The method supports videos of arbitrary length while avoiding scale misalignment and redundant computation caused by overlapping windows. Built upon a Vision Transformer encoder and a Dense Prediction Transformer head, the architecture enables efficient inference. Experimental results demonstrate that the proposed approach achieves state-of-the-art performance across multiple benchmarks, significantly outperforms existing methods in temporal consistency, and delivers a 2× speedup in practical inference speed.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Mixture of Experts (MoE)
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• Web Mining and Content Analysis: Large pretrained models with web data
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Applying single image Monocular Depth Estimation (MDE) models to video sequences introduces significant temporal instability and flickering artifacts. We propose a novel approach that adapts any state-of-the-art image-based (depth) estimation model for video processing by integrating a new temporal module - trainable on a single GPU in a few days. Our architecture StableDPT builds upon an off-the-shelf Vision Transformer (ViT) encoder and enhances the Dense Prediction Transformer (DPT) head. The core of our contribution lies in the temporal layers within the head, which use an efficient cross-attention mechanism to integrate information from keyframes sampled across the entire video sequence. This allows the model to capture global context and inter-frame relationships leading to more accurate and temporally stable depth predictions. Furthermore, we propose a novel inference strategy for processing videos of arbitrary length avoiding the scale misalignment and redundant computations associated with overlapping windows used in other methods. Evaluations on multiple benchmark datasets demonstrate improved temporal consistency, competitive state-of-the-art performance and on top 2x faster processing in real-world scenarios.

Problem

• temporal instability
• monocular depth estimation
• video depth estimation
• flickering artifacts

Innovation

• temporal stability
• monocular depth estimation
• cross-attention
• video depth estimation
• efficient inference