DrivingDepth: Sparse-Prompted Pixel-wise Scale Correction for Driving Depth Estimation

2026-06-30

AI Summary

This work addresses the challenge of fusing metrically inaccurate but structurally coherent dense geometry from pretrained foundation models with sparse, noisy LiDAR points in autonomous driving. The authors propose a novel paradigm that freezes the pretrained depth foundation model and leverages sparse LiDAR measurements as geometric prompts to learn only per-pixel scale residuals, thereby rectifying relative depth predictions into metric space without compromising the original geometric consistency. This approach uniquely unifies high-fidelity metric accuracy with robust visual geometric structure. Evaluated on the nuScenes benchmark under a four-view setting, the method achieves an AbsRel of 11.19 and an EdgeCR of 5.741, significantly outperforming existing approaches and establishing new state-of-the-art performance in both metric precision and geometric coherence.

Abstract

Dense depth estimation for autonomous driving faces a geometry-scale conflict: depth foundation models deliver pixel-aligned dense visual geometry without reliable metric scale, while projected LiDAR provides metric anchors that are sparse, noisy, and misaligned with image structures. Existing sparse-prompted methods incorporate LiDAR by regenerating depth from scratch, overriding the foundation model's coherent geometry and producing structural artifacts on visually continuous surfaces. Our key insight is that foundation models already capture geometrically coherent relative depth; no additional surface structure learning is required-only a per-pixel scale factor mapping relative geometry to metric coordinates. Based on this, we propose DrivingDepth, which treats sparse LiDAR as geometric prompts that locally calibrate a frozen foundation prior through residual pixel-wise scale correction, preserving dense visual geometry by construction. On nuScenes with 4-frame surround-view input, DrivingDepth achieves an AbsRel of 11.19 and an EdgeCR of 5.741, outperforming MapAnything (11.99/1.914) by simultaneously delivering SOTA metric accuracy and geometric consistency.

Problem

• depth estimation
• autonomous driving
• geometry-scale conflict
• LiDAR
• foundation models

Innovation

• scale correction
• sparse prompting
• depth estimation
• foundation model
• geometric consistency