Search papers, labs, and topics across Lattice.
The KAM-WM framework leverages a frozen latent video world model to derive Kinematic Affordance Maps (KAMs), which capture both where and how to interact with objects in robot manipulation tasks. By querying a Flow Matching image-to-video backbone, KAM-WM generates directional interaction cues without the need for rollout or fine-tuning, leading to significant performance improvements on manipulation benchmarks. The approach achieved an average success rate of 90.6% on LIBERO and varying success rates on RoboTwin2.0, highlighting the effectiveness of incorporating directional information into manipulation strategies.
Extracting interaction cues from a frozen video model enables robots to achieve up to 90.6% success in manipulation tasks without costly rollout processes.
Learning manipulation from few demonstrations requires visual priors that capture not only where to interact, but also how the interaction should begin; static priors such as segmentation masks encode only the former. We present KAM-WM, a framework that extracts a coarse directional interaction cue from a frozen latent video world model without rollout or world-model fine-tuning. KAM-WM queries a Flow Matching image-to-video backbone once and interprets its single-step latent velocity as a Kinematic Affordance Map (KAM), which provides task-conditioned interaction regions and coarse motion structure. A lightweight Perceiver compresses KAM into tokens that condition a diffusion policy together with RGB observations and proprioception. Across LIBERO and RoboTwin2.0, KAM-WM reaches 90.6% average success on LIBERO and achieves 65.7% and 22.4% success rates in the Easy and Hard settings on RoboTwin2.0, respectively. Controlled comparisons against a zero-order mask prior suggest that part of the gains comes from directional information beyond spatial localization alone. These results indicate that, in the evaluated settings, a frozen video model can provide a useful first-order visual prior for control without the test-time cost of future rollout.
Tsinghua AI