Search papers, labs, and topics across Lattice.
This paper introduces EgoGVAE, a novel approach for reconstructing full-body meshes using only head pose information, which is crucial for applications involving head-mounted devices. By employing a guided variational autoencoder, the method aligns latent distributions from a head-to-motion network with those of a guidance network, facilitating the decoding of full-body poses from minimal input. The results demonstrate that EgoGVAE achieves over 50 times faster inference compared to traditional diffusion-based methods while significantly enhancing reconstruction performance on benchmark datasets.
Recovering full-body meshes from head poses is now over 50 times faster without sacrificing quality, thanks to EgoGVAE's innovative approach.
We address the problem of recovering the full-body mesh from only the head pose. This task has become essential for various applications based on head-mounted devices or smart glasses. The challenge of this task lies in estimating the pose information of unobserved body parts based solely on a single joint (i.e., head) trajectory. Several studies have begun to adopt head-conditioned generative models, however, such previous methods are costly and time-consuming due to the diffusion-based iterative process. As an alternative, we propose a simple yet novel method that leverages the latent space of the guidance network, which is designed as a variational autoencoder taking full-body poses as inputs. By enforcing latent distributions of this guidance network and our head-to-motion network to be similar, latent features sampled from the'guided'distribution, i.e., distribution learned in our head-to-motion network, can be reliably decoded for natural representations of full-body poses even only with the head pose. One important advantage of the proposed method is that one-step sampling scheme achieves remarkably fast inference (more than 50 times faster) compared to diffusion-based approaches. Experimental results on benchmark datasets show that the proposed method efficiently improves the performance of ego-body mesh reconstruction.
