Search papers, labs, and topics across Lattice.
This paper introduces PoLAR, a novel approach to latent action pretraining that separates transition extent and mode by imposing a radial structure on latent actions. By leveraging temporal offsets between observations as a proxy for transition extent, PoLAR encourages larger radii for actions derived from observations with greater temporal separation, effectively enhancing the representation of diverse transition modes. Experimental results demonstrate that PoLAR significantly improves policy performance in both simulated and real-world robot tasks compared to existing latent action methods.
By factoring transition extent and mode in latent actions, PoLAR achieves superior policy performance, revealing the critical role of latent action geometry in robot learning.
Latent action pretraining learns representations of visual change from pairs of observations, but existing methods typically encode each transition as a single unstructured representation that entangles transition extent and transition mode. We introduce Polar Latent Actions with Radial structure (PoLAR), which imposes a radial-direction structure on latent actions, encouraging radius to encode transition extent and direction to retain transition mode. PoLAR uses temporal offset between two observations as a weak proxy for transition extent, encouraging latent action from observation pairs separated by larger temporal gaps to occupy larger radii. We instantiate this structure in hyperbolic space, whose expanding volume with radius offers a natural fit for more diverse transition modes at larger extents. Across in-task and large-scale pretraining settings, PoLAR improves downstream policy performance in simulation and real-world robot experiments, outperforming latent action baselines and strong pretrained VLAs. These results suggest that the geometry of the latent action space is an important design choice for transferring visual pretraining to downstream robot policy learning.
