Track manipulation, locomotion and embodied systems.
Search papers, labs, and topics across Lattice.
ActiveWAM addresses the evidence-aware retain-acquire problem in active vision manipulation by jointly learning observation and manipulation policies within a unified world-action model. It leverages training-time inversion, constraining a frozen video prior with task-relevant source evidence and temporal changes, thus avoiding costly test-time inversion. The model achieves state-of-the-art performance on diverse benchmarks, including real-world kitchen tasks, by generating view-aware bimanual and pan/tilt actions.
A single world-action model learns to actively control both cameras and manipulators, solving the evidence-aware retain-acquire problem without needing test-time inversion.
Active vision manipulation requires a policy to control both its camera and its end-effectors, yet camera motion determines which evidence remains visible within finite observation windows. Acquiring a new view can displace task-critical cues, while retaining a view forgoes potentially useful observations. We formulate this as an evidence-aware retain--acquire problem and present ActiveWAM, a unified world--action model that learns observation and manipulation jointly. To this end, we propose training-time inversion which constrains a frozen video prior by task-bearing source evidence and visible temporal changes, eliminating the need for test-time inversion or candidate ranking. At deployment, the policy generates bimanual and pan/tilt actions-including stay and reacquisition behaviors-from view-aware history, and updates context from newly measured RGB observations. Future-video prediction serves as a co-training signal, while action generation requires neither future-video decoding nor optimal viewpoint annotations. We introduce RoboTwin-AV, a 50-task benchmark with executable pan/tilt control and automatically generated demonstrations. ActiveWAM improves TAVIS out-of-distribution success by up to 17.0 percentage points over the strongest baselines, achieves 20.0 additional points over Fast-WAM on RoboTwin-AV, and outperforms it by 26.7 points on real-world physical kitchen tasks.