Search papers, labs, and topics across Lattice.
This paper introduces the Latent Diffusion Policy (LDP), a two-stage framework designed to enhance robotic manipulation by simplifying the learning process in diffusion-based visuomotor policies. By employing a conditional variational autoencoder (CVAE) to shape the latent space, LDP decouples scene comprehension from trajectory generation, resulting in a smoother velocity field that improves learning efficiency from limited demonstrations. The method significantly outperforms existing approaches on coordination-intensive tasks and demonstrates effective transfer to real-world applications, showcasing its practical utility in robotic systems.
LDP achieves a dramatic improvement in robotic manipulation performance by decoupling scene understanding from trajectory generation, leading to smoother learning dynamics.
Diffusion-based visuomotor policies operating directly in raw action spaces conflate scene comprehension with trajectory generation within a single denoising process. The resulting velocity field must simultaneously encode scene information and generate precise trajectories, increasing learning complexity and limiting performance on tasks demanding precise temporal coordination across multiple arms. To simplify this joint learning problem, we introduce Latent Diffusion Policy (LDP), a two-stage framework performing flow matching in a deliberately shaped latent space. By absorbing scene understanding into an observation-conditioned CVAE encoder, LDP concentrates the conditional distribution of each observation. Consequently, the flow model avoids implicitly resolving scene-dependent structures; instead, it generates within a pre-concentrated distribution featuring a smoother velocity field, simplifying learning from limited demonstrations. Furthermore, to capture temporal dependencies among latent tokens, LDP trains with per-token diffusion forcing and employs staircase inference sampling to resolve the resulting distributional mismatch. We also propose reconstruction FID (rFID) as a lightweight proxy predicting downstream task success solely from latent space statistics. On coordination-intensive tasks from RoboTwin 2.0, LDP outperforms DP3 by a substantial margin and transfers effectively to real-world bimanual deployments.
