Search papers, labs, and topics across Lattice.
This paper introduces SLIM, a compact 0.5B-parameter latent interaction policy designed for robot manipulation that learns action-grounded predictive latents through self-supervised masked trajectory prediction. By focusing on compact representations and employing a Mixture-of-Transformers backbone, SLIM effectively captures future transitions and the actions responsible for observed changes, achieving performance on par with larger multimodal models. The key result shows that SLIM not only matches the performance of larger vision-language-action (VLA) models but does so with significantly reduced computational resources and latency, making it a more efficient solution for real-time robot manipulation tasks.
SLIM achieves state-of-the-art performance in robot manipulation with just 0.5B parameters, outperforming larger models while slashing GPU memory usage and inference latency.
Vision-language-action policies rely on large multimodal backbones to jointly perform perception, language conditioning, and action generation at every control step. Much of this capacity supports open-domain semantics, whereas continuous robot manipulation primarily requires compact representations of observations, actions, and the transitions induced by actions. Pixel-level world models provide another route, but predicting visual details irrelevant to control can be unnecessarily expensive. We propose SLIM (Self-supervised Latent Interaction Model), a compact 0.5B-parameter latent interaction policy. SLIM learns action-grounded predictive latents that capture both action-conditioned future transitions and the actions that explain observed changes. SLIM learns these representations through self-supervised masked trajectory prediction, combining action reconstruction with future-latent prediction. A compact Mixture-of-Transformers (MoT) backbone models interactions between observation latents and action tokens. The resulting policy is trained with flow matching for language-conditioned action generation. Across simulation benchmarks and real-world evaluation, SLIM matches or exceeds representative large-scale VLA and world-action-model baselines with fewer parameters, no additional embodied pretraining, lower inference latency, and substantially lower GPU memory usage.
Tsinghua AI