Search papers, labs, and topics across Lattice.
This paper introduces DLAM, a distributional latent-action model that utilizes diagonal Gaussians to represent transitions in vision-language-action tasks, addressing the limitations of existing methods that suffer from deterministic transition points. By conditioning reconstruction on reference frames and employing normalized composition and reversal techniques, DLAM effectively captures temporal dynamics and reduces error propagation in recursive compositions. The model outperforms existing baselines in terms of temporal consistency and policy performance across multiple manipulation tasks, demonstrating significant improvements in both reconstruction and control capabilities.
DLAM achieves superior temporal consistency and policy performance by modeling transitions as distributional latent actions, fundamentally changing how we approach action generation in VLA tasks.
Vision-language-action (VLA) models remain constrained by scarce action-labeled robot data, whereas action-free videos offer abundant observations of physical change. Latent action models can extract such priors, but reconstruction-trained codes may predict future observations without the structure required for joint generation with robot actions. Existing structured methods add temporal constraints but retain deterministic transition points, so residual errors in locally inferred transitions may propagate and compound under recursive composition. We introduce DLAM, a distributional latent-action model that represents each transition as a diagonal Gaussian. Reconstruction conditioned on the reference frame grounds the mean in observed visual change, while normalized composition and reversal over equal-gap triplets constrain both the mean and dimension-wise variance. Variance composition uses a lightweight shared-correlation coefficient to account for dependence between adjacent transitions that share an intermediate frame, whereas reversal negates the mean and preserves the variance. For downstream policy learning, we freeze the encoder and train a flow-matching policy to jointly generate mean transition sequences and robot actions. On held-out transitions, DLAM learns more temporally consistent latent dynamics than existing latent-action baselines and achieves stronger direct and cumulative reconstruction on held-out videos. Under the same controlled $\pi_0$ transfer protocol, it also improves policy performance on MetaWorld MT50, LIBERO, and real-world manipulation tasks. Controlled ablations show that normalized mean constraints account for most of the reconstruction gain, while learned variance and correlation-aware composition provide complementary improvements in downstream control.
