Search papers, labs, and topics across Lattice.
This paper introduces a Joint Embedding Predictive Architecture (JEPA) for real-time control of quadrotors, addressing the limitations of traditional autoregressive models in long-horizon forecasting. By integrating a latent dynamics model with a physics-inspired prober, the authors achieve accurate long-horizon predictions and robust zero-shot sim-to-real transfer. Extensive experiments validate the model's effectiveness in diverse operating conditions, highlighting its potential for agile aerial vehicle control under uncertainty.
Achieving robust zero-shot sim-to-real transfer for quadrotors, this work redefines the boundaries of long-horizon prediction in robotic control.
Accurate dynamics models are critical for informed decision-making in robotic systems, particularly for agile aerial vehicles operating under uncertainty. Neural network dynamics models are attractive for capturing complex nonlinear effects, but existing predictive approaches struggle with long-horizon forecasting because their autoregressive rollout mechanism amplifies errors over time. Joint Embedding Predictive Architectures (JEPAs) offer a compelling alternative by modeling dynamics in latent space, yet prior JEPA-style methods for robot navigation have been studied primarily for kinematic-level planning, with limited investigation in high-frequency control. In this work, we introduce the JEPA-style model for real-time quadrotor control. The proposed approach combines a latent dynamics model with a novel physics-inspired prober that maps frozen latents to interpretable state, enabling physically grounded long-horizon prediction. Additionally, we combine the learned model with a sampling-based optimal control solution to take advantage of its predictive capabilities for real-time control on embedded hardware. Finally, to reduce the dependence on expensive and unsafe real-world data collection, we develop a structured pipeline for automated dataset generation. Extensive open-loop and outdoor closed-loop experiments demonstrate accurate prediction, robust zero-shot sim-to-real transfer, and strong generalization across diverse operating conditions.
Meta AI