Search papers, labs, and topics across Lattice.
This paper explores the integration of learned object perception into single-view 3D reconstruction, revealing that perceptual signals from pretrained models can significantly enhance reconstruction quality. By employing a model-agnostic approach, the authors demonstrate that their method can be seamlessly incorporated into existing reconstruction frameworks. Experiments show consistent and substantial improvements across two state-of-the-art pipelines, underscoring the effectiveness of merging perception with generative tasks.
Leveraging object perception can transform single-view 3D reconstruction, yielding substantial quality gains across various methods.
The relationship between object perception and reconstruction is well established in human vision, yet remains underexplored in computer vision. In this paper, we demonstrate that learnt object perception can significantly enhance 3D reconstruction. Focusing on the challenging task of single-view 3D object reconstruction, we propose a method that leverages perceptual signals extracted from pretrained perception models capturing semantic and geometric information to drive the reconstruction of an object from its single image. Our approach is model-agnostic and can be integrated into various reconstruction methods in a plug-and-play manner. Experiments with two state-of-the-art single-view 3D reconstruction pipelines in a benchmark dataset show consistent and substantial improvements achieved by our method, validating the effectiveness of incorporating perception into generation. We provide in-depth analysis of various aspects of our method and its application. Our project page is at https://ynhuhuynh.github.io/perception-3d/.
