Search papers, labs, and topics across Lattice.
This dissertation introduces VarIS, a custom light sphere designed to capture high-resolution stereo geometry and reflectance maps for creating re-topologized 3D facial meshes, significantly enhancing the efficiency of facial animation production. By integrating a deep learning framework that predicts facial meshes from synthetic multiview images generated by an in-house rendering system, the study achieves standardized meshes suitable for rigging with minimal human intervention. Key findings indicate that precise camera parameters and 3D landmark regularization substantially enhance landmark accuracy and geometric consistency, leading to improved reconstruction quality.
VarIS revolutionizes facial mesh capture, enabling photorealistic results with minimal human oversight and streamlined production workflows.
Creating re-topologized 3D facial meshes is essential for high-quality facial animation but remains labor-intensive and time-consuming. This dissertation explores more efficient approaches for capturing production-ready facial meshes through: (1) the development of VarIS, a custom light sphere for capturing high-resolution stereo geometry and reflectance maps; (2) analysis of camera parameters affecting automatic 2D and 3D landmarking; (3) synthetic-data methods for training neural face regression; and (4) techniques for improving neural multi-view face-shape regression. While VarIS enables photorealistic face capture, its operational and processing costs motivate a more scalable approach. A deep learning framework is therefore proposed to directly predict re-topologized facial meshes from synthetic multiview images generated with Visage Craft, an in-house physically based rendering system using an Appearance 3D Morphable Model (A3DMM). The system produces standardized meshes ready for rigging and animation with minimal human supervision. Results show that incorporating accurate camera intrinsics and extrinsics improves landmark accuracy and geometric consistency, while 3D landmark regularization further improves reconstruction quality.
