Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
The paper introduces PromptStereo, a zero-shot stereo matching approach that leverages monocular depth foundation models by incorporating structure and motion cues as prompts within a novel Prompt Recurrent Unit (PRU). PRU, built upon the decoder of a monocular depth foundation model, enriches latent representations with stereo-scale information while preserving monocular depth priors. Experiments demonstrate state-of-the-art zero-shot generalization performance across multiple datasets, highlighting the effectiveness of prompt-guided iterative refinement.
Forget feature extraction, PromptStereo shows how injecting structure and motion cues into a monocular depth model's decoder blows away zero-shot stereo matching benchmarks.
Modern stereo matching methods have leveraged monocular depth foundation models to achieve superior zero-shot generalization performance. However, most existing methods primarily focus on extracting robust features for cost volume construction or disparity initialization. At the same time, the iterative refinement stage, which is also crucial for zero-shot generalization, remains underexplored. Some methods treat monocular depth priors as guidance for iteration, but conventional GRU-based architectures struggle to exploit them due to the limited representation capacity. In this paper, we propose Prompt Recurrent Unit (PRU), a novel iterative refinement module based on the decoder of monocular depth foundation models. By integrating monocular structure and stereo motion cues as prompts into the decoder, PRU enriches the latent representations of monocular depth foundation models with absolute stereo-scale information while preserving their inherent monocular depth priors. Experiments demonstrate that our PromptStereo achieves state-of-the-art zero-shot generalization performance across multiple datasets, while maintaining comparable or faster inference speed. Our findings highlight prompt-guided iterative refinement as a promising direction for zero-shot stereo matching.