Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces Latent Sequence Optimization (LSO), a method that enhances the capabilities of Behavioral Foundation Models (BFMs) for precise motion tracking by optimizing a sequence of latent representations. By integrating simulation rollouts with policy gradient updates, LSO effectively addresses the limitations of existing heuristics that struggle with dynamic motions. The approach is validated through various applications, including dense tracking and real humanoid robot deployment, demonstrating significant improvements in motion tracking accuracy without the need for reward engineering.
LSO transforms BFMs into powerful tools for precise motion tracking, overcoming the limitations of traditional heuristics in dynamic environments.
Behavioral Foundation Models (BFMs) offer a promising path toward universal physics-based character control by organizing a rich repertoire of physically plausible behaviors into a latent space, guided by a large-scale motion dataset. While these models excel at time-invariant tasks, such as goal-reaching and state-based reward optimization, their latent space does not directly support time-varying objectives, such as tracking a motion sequence. For tracking, existing heuristics rely on moving-window-averaging that fails to capture the nuances of highly dynamic motions. In this work, we propose a novel Latent Sequence Optimization (LSO) to address these shortcomings. Our approach combines simulation rollouts with a policy gradient update to optimize over a sequence of latents, extending the capabilities of BFMs toward precise motion tracking without requiring reward engineering and tuning. To guide the optimization toward smooth, coherent latent trajectories, we model the latent sequence using temporally correlated noise. We validate our approach across dense tracking, sparse keyframing, and direct deployment onto a real humanoid robot.