TanDepth: Leveraging Global DEMs for Metric Monocular Depth Estimation in UAVs

2024-09-08

IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing

AI Summary

To address the scale ambiguity inherent in monocular depth estimation for unmanned aerial vehicles (UAVs), this paper proposes TanDepth—a plug-and-play, inference-time scale recovery method requiring no model retraining. TanDepth integrates sparse geographic priors from the Global Digital Elevation Model (GDEM) with estimated camera poses, enabling end-to-end, model-agnostic metric depth reconstruction via geometric projection matching and an enhanced cloth filter. It is the first approach to synergistically combine GDEM-derived elevation priors with physics-inspired filtering, substantially improving scale consistency. To facilitate rigorous evaluation, we construct and publicly release UAVid-Depth, the first UAV-oriented benchmark dataset extended specifically for depth estimation. Extensive experiments demonstrate that TanDepth reduces AbsRel error by 32% across diverse real-world UAV scenarios, outperforming existing UAV-adapted scale correction methods—without relying on auxiliary sensors or annotated ground-truth depth.

Technology Category

• Intelligent Robots: State Estimation
• Computer Vision: Low Level & Physics-based Vision
• Planning, Routing, and Scheduling: Replanning and Plan Repair

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Security and Privacy: Large-scale security measurements
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

Aerial scene understanding systems face stringent payload restrictions and must often rely on monocular depth estimation for modeling scene geometry, which is an inherently ill-posed problem. Moreover, obtaining accurate ground truth data required by learning-based methods raises significant additional challenges in the aerial domain. Self-supervised approaches can bypass this problem, at the cost of providing only up-to-scale results. Similarly, recent supervised solutions which make good progress toward zero-shot generalization also provide only relative depth values. This work presents TanDepth, a practical scale recovery method for obtaining metric depth results from relative estimations at inference-time, irrespective of the type of model generating them. Tailored for uncrewed aerial vehicle (UAV) applications, our method leverages sparse measurements from Global Digital Elevation Models (GDEM) by projecting them to the camera view using extrinsic and intrinsic information. An adaptation to the cloth simulation filter is presented, which allows selecting ground points from the estimated depth map to then correlate with the projected reference points. We evaluate and compare our method against alternate scaling methods adapted for UAVs, on a variety of real-world scenes. Considering the limited availability of data for this domain, we construct and release a comprehensive, depth-focused extension to the popular UAVid dataset to further research.

Problem

• Monocular depth estimation in UAVs
• Scale recovery from relative depth
• Integration of Global DEMs for accuracy

Innovation

• TanDepth for metric depth recovery
• Leverages Global DEMs in UAVs
• Adapts Cloth Simulation Filter