Search papers, labs, and topics across Lattice.
This paper introduces FoundDP, a unified framework that enhances dual-pixel depth estimation by integrating metric depth with global structural priors from a monocular depth foundation model. The approach addresses the limitations of existing methods that struggle with weak disparity signals, particularly in textureless or low-contrast regions, by employing Vision Transformer features to restore structural consistency. Extensive experiments demonstrate that FoundDP significantly improves both structural fidelity and metric accuracy, particularly in challenging conditions with reduced disparity observability.
FoundDP transforms dual-pixel depth estimation by combining local metric cues with global structural insights, achieving unprecedented accuracy in low-visibility scenarios.
Dual-pixel (DP) imaging enables metric depth estimation from a single camera using sub-aperture disparity. However, the extremely small effective baseline limits disparity observability, leading to structural degradation and depth failure in textureless, low-contrast, or downsampled regions. Existing DP-based methods rely primarily on local disparity cues and therefore become unreliable when disparity signals are weak or ambiguous. To address this limitation, we propose \emph{FoundDP}, a unified framework that integrates metric DP depth with global structural priors from a monocular depth foundation model. Our method preserves metric scale through DP-derived depth and leverages Vision Transformer (ViT) features to restore structural consistency in weak-disparity regions. To ensure reliable metric guidance under DP imaging conditions, we identify and mitigate ViT representation degradation induced by DP defocus blur via ViT feature alignment, enabling stable metric-guided depth estimation. Extensive experiments on synthetic and real-world DP benchmarks show that FoundDP delivers superior performance, with consistent gains in structural fidelity and metric accuracy, especially under reduced disparity observability. Code will be available at: https://github.com/EchoLighting/FoundDP