Search papers, labs, and topics across Lattice.
UVFaceFusion introduces a novel feed-forward framework for fast and accurate multi-view face reconstruction that maintains topological consistency, addressing the trade-offs between geometric fidelity and generalization in real-world scenarios. By leveraging learnable neural fusion in a canonical UV space, the method effectively integrates dense point maps and facial UV correspondences from multiple views, resulting in a high-fidelity mesh reconstruction. The approach achieves state-of-the-art performance, reconstructing a mesh from 16 input views in under 3 seconds on a single RTX 4090, demonstrating its efficiency and robustness across diverse datasets.
Achieving high-fidelity face reconstruction in under 3 seconds, UVFaceFusion redefines the balance between speed and accuracy in digital avatar creation.
Reconstructing high-fidelity facial geometry with an assigned topology is essential for digital avatar creation and animation, yet existing automated methods often trade off geometric fidelity and in-the-wild generalization. We present UVFaceFusion, a feed-forward framework for multi-view, fixed-topology face reconstruction from daily images. Our key idea is to replace heuristic topological optimization with learnable neural fusion in a canonical UV space. Given multi-view images, we first obtain dense point maps and facial UV correspondences of each view using VGGT and Pixel3DMM, respectively. Then, the view-specific point maps are lifted into the canonical UV domain and fused with a novel mask-aware neural fusion network. The network predicts a complete UV-space point map, from which a fixed-topology mesh is directly sampled. Although trained only on Ava-256, UVFaceFusion generalizes well to multiple public benchmarks and in-the-wild captures, benefiting from its canonical UV-space geometry-to-geometry fusion that reduces dependence on dataset-specific appearance and capture conditions. Experiments on various benchmarks show that UVFaceFusion achieves state-of-the-art reconstruction accuracy while reconstructing a mesh from 16 input views in less than 3 seconds on a single RTX 4090. Code is available at https://github.com/grignarder/UVFaceFusion.
Tsinghua AI