Search papers, labs, and topics across Lattice.
The paper introduces HumanNOVA, a novel model for generating photorealistic 3D human avatars from a single RGB image, addressing the challenges posed by the limited availability of diverse 3D human data. By leveraging a scalable data generation pipeline that combines rigged assets with multi-camera captures, the authors significantly enhance the quantity and diversity of training data, enabling robust model performance. HumanNOVA employs a feed-forward, token-conditioned architecture that allows for rapid inference, achieving superior results across multiple benchmarks without the need for test-time optimization.
Achieving photorealistic 3D human avatars from a single image in under a second could revolutionize virtual reality and gaming applications.
In this paper, we present HumanNOVA, a photorealistic, universal, and rapid model for generating 3D human avatars from a single RGB image. Achieving both photorealism and generalization is challenging due to the scarcity of diverse, high-quality 3D human data. To address this, we build a scalable data generation pipeline that follows two strategies. The first one is to leverage existing rigged assets and animate them with extensive poses from daily life. The second strategy is to utilize existing multi-camera captures of humans and employ fitting to generate more diverse views for training. These two strategies enable us to scale up to 100k assets, significantly enhancing both the quantity and the diversity of data for robust model training. In terms of the architecture, HumanNOVA adopts a feed-forward, token-conditioned avatar modeling framework that allows fast inference in less than one second and requires no test-time optimization. Given an input image and an estimated simplified human mesh (SMPL) without detailed geometry or appearance, the model first encodes both inputs into compact token representations. These tokens then act as conditioning signals and are fused through cross-attention to construct a triplane-based 3D avatar representation. Extensive experiments on multiple benchmarks demonstrate the superiority of our approach, both quantitatively and qualitatively, as well as its robustness under diverse input image conditions. Project page at https://HumanNOVA.github.io .
NUS