Need for Speed: Zero-Shot Depth Completion with Single-Step Diffusion

2026-03-11

AI Summary

This work addresses the inefficiency of conventional diffusion models in depth completion, which rely on test-time optimization and thus suffer from slow inference that hinders low-latency applications. To overcome this limitation, the authors propose Marigold-SSD, a single-step, post-fusion depth completion framework that shifts the computational burden to the fine-tuning stage, thereby enabling fast inference while preserving the strong generative priors of diffusion models. Marigold-SSD is the first diffusion-based approach to achieve single-step depth completion without any test-time optimization. It demonstrates state-of-the-art zero-shot cross-domain performance across four indoor and two outdoor benchmarks, while requiring only 4.5 GPU-days for training—significantly narrowing the efficiency gap between diffusion models and discriminative methods.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Large Multimodal Models (LMMs)
• Search and Optimization: Learning to Search

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines

Abstract

We introduce Marigold-SSD, a single-step, late-fusion depth completion framework that leverages strong diffusion priors while eliminating the costly test-time optimization typically associated with diffusion-based methods. By shifting computational burden from inference to finetuning, our approach enables efficient and robust 3D perception under real-world latency constraints. Marigold-SSD achieves significantly faster inference with a training cost of only 4.5 GPU days. We evaluate our method across four indoor and two outdoor benchmarks, demonstrating strong cross-domain generalization and zero-shot performance compared to existing depth completion approaches. Our approach significantly narrows the efficiency gap between diffusion-based and discriminative models. Finally, we challenge common evaluation protocols by analyzing performance under varying input sparsity levels. Page: https://dtu-pas.github.io/marigold-ssd/

Problem

• depth completion
• diffusion models
• zero-shot
• inference efficiency
• 3D perception

Innovation

• single-step diffusion
• zero-shot depth completion
• late-fusion
• efficient 3D perception
• cross-domain generalization