PromptStereo: Zero-Shot Stereo Matching via Structure and Motion Prompts

2026-03-02

AI Summary

Existing zero-shot stereo matching methods struggle to effectively integrate monocular depth priors during iterative refinement, limiting their generalization performance. This work proposes the Prompt Recurrent Unit (PRU), which introduces a prompt-based mechanism into the iterative optimization phase for the first time. By leveraging structural and motion cues as prompts injected into the decoder of a monocular depth foundation model, PRU enables absolute-scale-aware feature updates. This approach transcends the limitations of conventional GRU architectures, substantially enhancing representational capacity. As a result, it achieves state-of-the-art zero-shot generalization across multiple benchmarks while maintaining comparable or even faster inference speed.

Technology Category

• Computer Vision: Other Foundations of Computer Vision
• Search and Optimization: Learning to Search
• Machine Learning: Deep Generative Models & Autoencoders

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Modern stereo matching methods have leveraged monocular depth foundation models to achieve superior zero-shot generalization performance. However, most existing methods primarily focus on extracting robust features for cost volume construction or disparity initialization. At the same time, the iterative refinement stage, which is also crucial for zero-shot generalization, remains underexplored. Some methods treat monocular depth priors as guidance for iteration, but conventional GRU-based architectures struggle to exploit them due to the limited representation capacity. In this paper, we propose Prompt Recurrent Unit (PRU), a novel iterative refinement module based on the decoder of monocular depth foundation models. By integrating monocular structure and stereo motion cues as prompts into the decoder, PRU enriches the latent representations of monocular depth foundation models with absolute stereo-scale information while preserving their inherent monocular depth priors. Experiments demonstrate that our PromptStereo achieves state-of-the-art zero-shot generalization performance across multiple datasets, while maintaining comparable or faster inference speed. Our findings highlight prompt-guided iterative refinement as a promising direction for zero-shot stereo matching.

Problem

• zero-shot stereo matching
• iterative refinement
• monocular depth priors
• stereo matching
• generalization

Innovation

• Prompt Recurrent Unit
• zero-shot stereo matching
• monocular depth prior
• iterative refinement
• structure and motion prompts