Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper presents FaceSnap, an innovative framework that enhances real-time facial performance capture by utilizing a two-stage approach to create a personalized model from a range-of-motion sequence. By enabling high-fidelity capture from a single monocular lightstage camera, FaceSnap significantly reduces the resource and labor demands associated with traditional multi-camera setups. The system achieves competitive geometric accuracy and dynamic 4K texture capture at 83 fps, surpassing existing methods that rely on extensive multi-view data.
Achieving production-quality facial performance capture from a single camera, FaceSnap redefines the efficiency of digital human generation.
Lightstage facial capture produces production-quality digital humans, but it is resource and labor-intensive. Multi-camera setups, hours of computation, and massive data storage create bottlenecks that hinder iterative workflows. This paper introduces FaceSnap, an end-to-end framework that streamlines capture via a two-stage approach. First, a one-time multi-view optimization from a range-of-motion sequence builds a personalized model encoding both geometry and expression-dependent appearance. This model then enables high-fidelity real-time facial performance capture from a single monocular lightstage camera, with no further multi-view capture required. FaceSnap jointly estimates geometry and dynamic 4K texture at 83 fps. The 4K texture is produced by a novel personalized residual upscaler that recovers subject-specific high-frequency detail, which generic upscalers fail to capture. FaceSnap achieves geometric accuracy competitive with full per-frame multi-view optimization while outperforming feed-forward methods trained on production-quality 3D data, all from a single camera view. Finally, we introduce Multi4D, a public benchmark for evaluating 4D facial reconstruction methods in lightstage environments, enabling topology-invariant geometric comparison across methods.