Search papers, labs, and topics across Lattice.
This paper introduces a novel approach to 4D facial reconstruction from image sequences by predicting canonical facial point coordinates, effectively transforming dynamic reconstruction into a canonical reconstruction problem. A transformer-based model is trained to jointly predict depth and canonical coordinates using multi-view geometry data warped into a canonical space. The method achieves state-of-the-art performance on reconstruction and tracking tasks, demonstrating significantly reduced correspondence error and improved depth accuracy compared to existing dynamic reconstruction techniques.
Achieve 3x faster and more accurate 4D facial reconstruction by predicting canonical facial point coordinates, enabling temporally consistent geometry and reliable correspondences within a single feed-forward model.
Accurate reconstruction and tracking of dynamic human faces from image sequences is challenging because non-rigid deformations, expression changes, and viewpoint variations occur simultaneously, creating significant ambiguity in geometry and correspondence estimation. We present a unified method for high-fidelity 4D facial reconstruction based on canonical facial point prediction, a representation that assigns each pixel a normalized facial coordinate in a shared canonical space. This formulation transforms dense tracking and dynamic reconstruction into a canonical reconstruction problem, enabling temporally consistent geometry and reliable correspondences within a single feed-forward model. By jointly predicting depth and canonical coordinates, our method enables accurate depth estimation, temporally stable reconstruction, dense 3D geometry, and robust facial point tracking within a single architecture. We implement this formulation using a transformer-based model that jointly predicts depth and canonical facial coordinates, trained using multi-view geometry data that non-rigidly warps into the canonical space. Extensive experiments on image and video benchmarks demonstrate state-of-the-art performance across reconstruction and tracking tasks, achieving approximately 3$\times$ lower correspondence error and faster inference than prior dynamic reconstruction methods, while improving depth accuracy by 16%. These results highlight canonical facial point prediction as an effective foundation for unified feed-forward 4D facial reconstruction.
