Zero-shot Depth Completion via Test-time Alignment with Affine-invariant Depth Prior

2025-02-10

AI Summary

Depth completion is an ill-posed problem of reconstructing dense depth maps from sparse measurements; existing methods rely on domain-specific learned priors, resulting in poor cross-domain generalization. This paper proposes the first zero-shot cross-domain depth completion framework: it introduces a pre-trained, affine-invariant depth diffusion model as a universal prior and, at test time, aligns its output to the metric scale of input sparse measurements via an optimization process incorporating hard-constraint projection. Crucially, no training data from the target domain is required. The method significantly enhances cross-domain generalizability while preserving geometric fidelity. Evaluated on multiple cross-domain benchmarks, it achieves an average 21% reduction in error over prior approaches, yields sharper structural details, and improves spatial understanding—demonstrating superior robustness and consistency across diverse unseen domains.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Deep Generative Models & Autoencoders
• Search and Optimization: Non-convex Optimization

Application Category

• Search and Retrieval-Augmented AI: Vertical and domain-specific search
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Depth completion, predicting dense depth maps from sparse depth measurements, is an ill-posed problem requiring prior knowledge. Recent methods adopt learning-based approaches to implicitly capture priors, but the priors primarily fit in-domain data and do not generalize well to out-of-domain scenarios. To address this, we propose a zero-shot depth completion method composed of an affine-invariant depth diffusion model and test-time alignment. We use pre-trained depth diffusion models as depth prior knowledge, which implicitly understand how to fill in depth for scenes. Our approach aligns the affine-invariant depth prior with metric-scale sparse measurements, enforcing them as hard constraints via an optimization loop at test-time. Our zero-shot depth completion method demonstrates generalization across various domain datasets, achieving up to a 21% average performance improvement over the previous state-of-the-art methods while enhancing spatial understanding by sharpening scene details. We demonstrate that aligning a monocular affine-invariant depth prior with sparse metric measurements is a proven strategy to achieve domain-generalizable depth completion without relying on extensive training data. Project page: https://hyoseok1223.github.io/zero-shot-depth-completion/.

Problem

• Generalize depth completion across domains.
• Align affine-invariant depth prior.
• Improve spatial understanding and detail.

Innovation

• Zero-shot depth completion
• Affine-invariant depth prior
• Test-time alignment optimization