Towards Depth Foundation Model: Recent Trends in Vision-Based Depth Estimation

2025-07-15

AI Summary

Existing visual depth estimation methods suffer from poor generalization, low stability, and reliance on small-scale, domain-specific training data. Method: This paper proposes the “Depth Foundation Model” paradigm—a unified framework designed for strong zero-shot cross-scene transferability. It integrates diverse input modalities (monocular, stereo, multi-view, and video sequences) and employs self-supervised and weakly supervised learning strategies to train a high-capacity, modular neural architecture on large-scale heterogeneous datasets. Contribution/Results: We formally define the Depth Foundation Model concept and its technical roadmap for the first time; develop a scalable training framework and standardized evaluation benchmark; and demonstrate significant improvements in robustness and accuracy on unseen scenes. The resulting model enables high-resolution, environment-robust, and cost-effective depth perception—advancing applications in 3D reconstruction, autonomous driving, and AR/VR.

Technology Category

• Machine Learning: Deep Neural Architectures and Foundation Models
• Computer Vision: Other Foundations of Computer Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Web Mining and Content Analysis: Large pretrained models with web data
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

Depth estimation is a fundamental task in 3D computer vision, crucial for applications such as 3D reconstruction, free-viewpoint rendering, robotics, autonomous driving, and AR/VR technologies. Traditional methods relying on hardware sensors like LiDAR are often limited by high costs, low resolution, and environmental sensitivity, limiting their applicability in real-world scenarios. Recent advances in vision-based methods offer a promising alternative, yet they face challenges in generalization and stability due to either the low-capacity model architectures or the reliance on domain-specific and small-scale datasets. The emergence of scaling laws and foundation models in other domains has inspired the development of "depth foundation models": deep neural networks trained on large datasets with strong zero-shot generalization capabilities. This paper surveys the evolution of deep learning architectures and paradigms for depth estimation across the monocular, stereo, multi-view, and monocular video settings. We explore the potential of these models to address existing challenges and provide a comprehensive overview of large-scale datasets that can facilitate their development. By identifying key architectures and training strategies, we aim to highlight the path towards robust depth foundation models, offering insights into their future research and applications.

Problem

• Developing cost-effective vision-based depth estimation methods
• Improving generalization and stability in depth estimation models
• Exploring large-scale datasets for robust depth foundation models

Innovation

• Develops depth foundation models for generalization
• Surveys deep learning architectures across settings
• Explores large-scale datasets for model training