Learning Image-Adaptive Scale Fields for Metric Depth Recovery

2026-05-08

AI Summary

Monocular depth estimation typically yields only relative scale, making it challenging to recover accurate metric depth when only sparse ground-truth scale anchors are available. This work proposes an image-adaptive scale field learning approach that extracts semantic and geometric cues from intermediate features of a monocular depth model to generate basis maps. Leveraging these bases, the method computes low-dimensional linear combination weights via least-squares optimization using the sparse anchors, enabling efficient and robust scale rectification. The proposed technique exhibits strong robustness even under extremely sparse anchoring conditions, provides an interpretable spatial representation of scale variation, and significantly improves metric depth accuracy across multiple datasets and state-of-the-art models, demonstrating its effectiveness and generalizability.

Abstract

Monocular depth estimation (MDE) typically produces depth estimations that are defined up to an unknown scale or shift. When only sparse metric anchors are available, recovering accurate metric depth becomes challenging yet necessary for practical applications. We address this problem by formulating metric depth recovery as image-adaptive scale field modeling. Instead of directly correcting the depth, we reformulate the correction as a low-dimensional linear combination of image-adaptive basis maps. These maps are derived from semantic and geometric cues encoded in the MDE estimations and intermediate representations. The weights of basis maps are efficiently determined from sparse metric anchors via a least-squares problem. This formulation yields improved metric depth accuracy, strong robustness under extreme anchor sparsity, and an interpretable decomposition of spatial scale variations. Extensive experiments across multiple datasets and representative MDE models demonstrate the effectiveness and general applicability of our approach.

Problem

• monocular depth estimation
• metric depth recovery
• scale ambiguity
• sparse anchors
• depth scaling

Innovation

• image-adaptive scale field
• metric depth recovery
• monocular depth estimation
• sparse metric anchors
• basis map decomposition