Search papers, labs, and topics across Lattice.
The paper introduces FiCA, a novel pipeline that generates lifelike avatars from a single portrait image using a combination of human-centric vision foundation models and a diffusion model. This approach tackles the challenge of limited visual information by employing a generative mapping to reconstruct 3D meshes and a feed-forward mesh refinement network to enhance fidelity and identity preservation without requiring person-specific optimization. Experimental results show that FiCA's avatars not only accurately represent diverse identities but also exceed the visual quality of those produced by existing methods.
FiCA generates photorealistic avatars from a single image, achieving unprecedented visual quality and identity fidelity without the need for individual optimization.
We introduce FiCA, a Feed-forward, instant Gaussian Codec Avatar generation pipeline that creates lifelike avatars from a single portrait image. Generating a photorealistic and drivable avatar from just a single image is significantly challenging due to the limited visual information available to accurately infer the 3D appearance and geometry of human heads. To address this, we develop a novel system that combines human-centric vision foundation models with a diffusion model. This system is designed to fully exploit partial visual observations to generate lifelike human avatars. Our proposed diffusion model learns a generative mapping from these partial observations to complete and authentic 3D mesh reconstruction. Additionally, we introduce a feed-forward mesh refinement network that enhances the fidelity and identity preservation of the generated avatars, eliminating the need for person-specific test-time optimization. By leveraging a universal prior model that decodes a generated mesh into a set of 3D Gaussians, we generate a photorealistic 3D Gaussian avatar, capable of being driven with novel expressions in real-time. Our experiments demonstrate that the avatars generated by our feed-forward approach faithfully represent diverse identities and surpass the visual quality of avatars produced by recent competing methods.
Meta AI
NVIDIA