Search papers, labs, and topics across Lattice.
This paper introduces SA4Depth, a novel approach for self-supervised monocular depth estimation that focuses on aligning the scene scales estimated by the pose and depth networks. The method refines pose estimates by reprojecting learnable visual features across frames using the estimated depth, minimizing feature alignment residuals. Experiments on KITTI, Cityscapes, and NYUv2 demonstrate that SA4Depth significantly improves depth estimation and pose consistency without increasing inference time.
Self-supervised depth estimation gets a boost: SA4Depth aligns scene scales between pose and depth networks, leading to substantial improvements in depth prediction without sacrificing inference speed.
Self-supervised depth estimation from monocular sequences relies on the joint learning of a depth and a pose network. Despite abundant research done to improve the depth network, efforts on the pose remain limited. In this context, even when depth is estimated up to scale, we highlight the importance of the alignment between the scene scales estimated by the pose and depth nets. Then, we introduce SA4Depth, an approach to improve this alignment and boost the depth predictions while keeping the inference time unchanged. Our proposed method uses the depth estimated during training to reproject learnable visual features across consecutive frames and refine the pose estimates by reducing feature alignment residuals. With our method, the estimated scene scales by the separate depth and pose networks are aligned, and the prediction scale consistency is improved across different sequences. Our differentiable refinement integrates seamlessly into existing self-supervised pipelines and substantially improves their depth estimates. We demonstrate this with extensive experiments both outdoors and indoors on KITTI, Cityscapes, and NYUv2. Additionally, results on KITTI Odometry confirm the effectiveness of our pose refinement. Our code is available at https://github.com/Runningchauncey/SA4Depth .
