Search papers, labs, and topics across Lattice.
This paper introduces a self-supervised framework for stereoscopic video generation that leverages monocular videos, addressing the critical bottleneck of stereo inpainting in Depth-Image-Based Rendering (DIBR). By applying the Geometric Reciprocity Theorem (GRT), the authors establish a relationship between disocclusion masks during target view synthesis and those lost when warping back to the source, allowing for analytical computation of disocclusion masks from monocular images. The proposed method significantly outperforms existing training-free and supervised techniques, demonstrating the potential for high-quality stereo content generation without the need for scarce stereo pairs or synthetic data.
Self-supervised learning from monocular videos can now achieve superior stereo video generation by directly computing disocclusion masks, eliminating reliance on expensive stereo datasets.
Monocular-to-stereo conversion synthesizes stereoscopic content from 2D videos for immersive 3D experiences. In modern Depth-Image-Based Rendering (DIBR) approaches, stereo inpainting of disocclusions is the critical bottleneck. Training-based methods achieve superior quality but rely on scarce stereo pairs or synthetic data with domain gaps. We address this through the first self-supervised framework learning from monocular videos via cycle consistency. Our key contribution is the Geometric Reciprocity Theorem (GRT): under the nearest-neighbor DIBR formulation, the disocclusion mask when synthesizing a target view equals the mask of pixels lost when warping back from target to source, enabling analytical computation of test-time disocclusion masks directly from monocular images. This yields train-test consistency for the stated warping formulation, supporting self-supervised learning from unlimited monocular videos and substantial improvements over training-free and supervised state-of-the-art methods. Project page: https://visual-ai.github.io/grt/
