Search papers, labs, and topics across Lattice.
The paper introduces LM-X, a novel framework for generalist robot manipulation that enhances long-horizon behavior through multi-timescale action prediction, incorporating task progress, event transitions, and uncertainty estimates. By emitting three supervised signals鈥攔eturn-to-go (RTG), event-to-go (ETG), and heteroscedastic action flow鈥擫M-X provides intrinsic explanations for control decisions, addressing the limitations of traditional action-only models. Experimental results demonstrate that LM-X significantly outperforms existing models, achieving a 74.1% success rate on complex tasks, highlighting the effectiveness of its predictive state representation in improving robotic control and interpretability.
Explicit multi-timescale predictions in LM-X not only enhance robot manipulation success rates but also provide intrinsic explanations for control decisions, transforming how we understand robotic action generation.
Generalist vision--language--action (VLA) policies learn long-horizon behavior mainly through short-horizon action prediction and reveal little beyond sampled commands. This creates two coupled bottlenecks: a single action target must implicitly absorb task progress, intermediate intent, and local reliability, while these control states remain hidden during execution. Inspired by functional principles of biological sensorimotor control, we introduce LM-X , which organizes prediction across task, event, and motor scales without claiming anatomical correspondence. Three explicitly supervised signals are emitted online and directly condition action generation: return-to-go (RTG) measures visible task progress, event-to-go (ETG) identifies the next semantic transition, and heteroscedastic action flow estimates local reliability through propagated variance. Explanation is therefore intrinsic to control rather than generated post hoc. Before a costly 20-day pretraining run on 64 NVIDIA B200 GPUs, a controlled five-task pretraining gate verifies the design: the complete model improves success by 16.0 points over the action-only backbone and by 10.8 points over the strongest single-head variant. We then train LM-X on more than 20,000 hours of real-robot trajectories, including over 1,000 hours of failed policy rollouts. LM-X achieves 74.1\% across 50 randomized-hard RoboTwin2.0 tasks versus 55.4\% for GR00T N1.7, and 68.6\% versus 50.7\% across seven real-robot tasks. RTG tracks semantic progress and visible regression, while variance rises during hesitation and oscillatory control. These results show that explicit multi-timescale predictive state can strengthen control while exposing interpretable internal estimates.
