Search papers, labs, and topics across Lattice.
This paper introduces EmbodiedVAE, a novel video Variational Autoencoder designed specifically for embodied manipulation tasks, addressing the limitations of existing latent diffusion models that struggle with compact and controllable representations. By employing a dual-encoder, single-decoder architecture alongside an asymmetric spatio-temporal compression module, EmbodiedVAE effectively disentangles robotic motion from environmental backgrounds, enhancing both representation efficiency and control precision. Experimental results show that EmbodiedVAE not only improves reconstruction quality with a 2dB PSNR increase over state-of-the-art methods but also facilitates fine-grained action control in robotic scenarios.
Achieving a 2dB PSNR improvement, EmbodiedVAE transforms how robots learn and execute manipulation tasks by providing compact and controllable latent representations.
Latent diffusion models (LDMs) have recently significantly advanced embodied learning in constructing powerful embodied manipulation world models. However, despite the remarkable performance, existing LDMs predominantly rely on Variational Autoencoders (VAEs) optimized for natural scenes while failing to account for the unique characteristics of embodied manipulation scenarios, yielding latent representations that are neither compact nor controllable, thereby hindering efficient training of LDMs and precise robotic control. To solve this problem, we present EmbodiedVAE, a novel video VAE that provides compact yet controllable latent representations tailored for the robotic manipulation world models. Specifically, EmbodiedVAE adopts a dual-encoder, single-decoder architecture with an asymmetric spatio-temporal compression module, which automatically disentangles the robot arm's motion from background environment, resulting in overall compactness while providing explicit embodied latent to support fine-grained action control. To further preserve the temporal consistency of learned robotic motion latent, we introduce an optimal-transport-based consistency module that explicitly enforces motion fidelity and inter-frame coherence. Extensive experiments demonstrate that our proposed EmbodiedVAE achieves superior reconstruction quality with high compression rate, while enabling more precise action control in robotic manipulation scenarios with an average of 2dB PSNR improvement over state-of-the-art video VAEs.
