Search papers, labs, and topics across Lattice.
This paper introduces RealSkin, a self-supervised framework that optimizes correspondences in a learned spectral domain to facilitate the transfer of visual attributes from real images to synthetic 3D surfaces. By employing a spatial-guided registration algorithm and a spectral-aware neural adjoint network, RealSkin effectively addresses the challenges of resolution mismatch and partial correspondences, which have hindered traditional methods. Experimental results show that RealSkin outperforms existing techniques in real-to-synthetic attribute transfer, achieving state-of-the-art results in challenging scenarios.
RealSkin bridges the gap between real-world images and synthetic 3D models by optimizing correspondences in a learned spectral domain, achieving unprecedented accuracy in attribute transfer.
Creating photorealistic 3D assets requires bridging the appearance gap between real-world observations and synthetic models. A promising approach is to transfer visual attributes from real images onto synthetic 3D surfaces. Traditional methods struggle with resolution mismatch and the inherent discreteness of point correspondences. In contrast, resolution-robust functional maps enable smooth attribute propagation but rely on near-isometry assumptions and topological consistency. To address these limitations, we propose RealSkin, a self-supervised framework that performs correspondence optimization in a learned spectral domain, guided by spatial correspondences. We first introduce a spatial-guided registration algorithm to establish coarse correspondences under severe topological discrepancies. To relax strict isometric assumptions and handle partial correspondences, we further design a spectral-aware neural adjoint network that incorporates partial correspondences into a neural function space and models non-isometric residuals for correspondence refinement. Experimental results demonstrate that our method achieves state-of-the-art performance on challenging real-to-synthetic scenarios. The code will be publicly released.
