Search papers, labs, and topics across Lattice.
This paper introduces a novel approach to fine-tuning a 3D-aware generative model using reinforcement learning from human feedback (RLHF) without relying on explicit surface representations or mesh priors. By directly optimizing the radiance-field density values with a learned reward model trained on a small set of preference samples, the method achieves significant improvements in 3D geometry quality. The results demonstrate that the fine-tuned model generates face geometries that align with user preferences in 74.4% of pairwise comparisons, despite a modest increase in distributional cost as measured by FID.
Fine-tuning a 3D GAN with human preferences directly on radiance fields yields face geometries that users overwhelmingly prefer, without the need for complex mesh extraction.
Reinforcement learning from human feedback (RLHF) for 3D generation is now established across a number of works, but most existing pipelines optimise explicit surface representations, often by converting radiance fields into meshes and training heavily on surface-supervised data. We instead fine-tune a pretrained 3D-aware generative model directly from a learned reward over radiance-field density ($蟽$) values, with no externally supplied mesh or shape prior. The reward model requires no pretraining, trains easily on a small set of preference samples, and yields robust improvement in 3D geometry. Working on an unconditional 3D-aware face GAN (EG3D), our reward reads the continuous 3D density field of the neural radiance field (NeRF) directly and supplies a geometry-only learning signal, requiring neither text conditioning, mesh extraction, nor multi-view rendering. A density-consistency constraint keeps the 2D appearance qualitatively similar while the geometry is reshaped, at a measurable but bounded distributional cost (FID-50k rises from 4.09 to 6.66): the fine-tuned generator, trained from the preferences of a single annotator as a proof of concept, produces face geometries preferred by users in 74.4% of pairwise comparisons.
