Efficient Test-Time Optimization for Depth Completion via Low-Rank Decoder Adaptation

2026-03-02

AI Summary

This work addresses the high computational cost and low inference efficiency of existing zero-shot depth completion methods during test-time optimization, which struggle to balance accuracy and speed. The authors propose a lightweight test-time optimization strategy that adaptively fine-tunes only a low-dimensional subspace within the decoder of a pretrained depth foundation model, guided by sparse depth supervision. This approach is motivated by the key observation that depth-relevant information is concentrated in a low-rank subspace of the decoder, thereby avoiding repeated full-network forward–backward passes. Evaluated on five indoor and outdoor datasets, the method achieves significantly improved inference efficiency while maintaining or even surpassing the accuracy of current state-of-the-art approaches, establishing a new Pareto frontier between precision and computational efficiency.

Technology Category

• Computer Vision: Learning & Optimization for CV
• Search and Optimization: Learning to Search
• Machine Learning: Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

Zero-shot depth completion has gained attention for its ability to generalize across environments without sensor-specific datasets or retraining. However, most existing approaches rely on diffusion-based test-time optimization, which is computationally expensive due to iterative denoising. Recent visual-prompt-based methods reduce training cost but still require repeated forward--backward passes through the full frozen network to optimize input-level prompts, resulting in slow inference. In this work, we show that adapting only the decoder is sufficient for effective test-time optimization, as depth foundation models concentrate depth-relevant information within a low-dimensional decoder subspace. Based on this insight, we propose a lightweight test-time adaptation method that updates only this low-dimensional subspace using sparse depth supervision. Our approach achieves state-of-the-art performance, establishing a new Pareto frontier between accuracy and efficiency for test-time adaptation. Extensive experiments on five indoor and outdoor datasets demonstrate consistent improvements over prior methods, highlighting the practicality of fast zero-shot depth completion.

Problem

• zero-shot depth completion
• test-time optimization
• computational efficiency
• depth completion
• inference speed

Innovation

• test-time adaptation
• depth completion
• low-rank decoder
• zero-shot learning
• efficient inference