Search papers, labs, and topics across Lattice.
This paper introduces EA-WM, an event-aware world model framework that enhances traditional visual-feature dynamics by incorporating task-specification-grounded event prediction and verification for long-horizon manipulation tasks. By rolling out candidate futures in a pretrained visual-feature space and scoring them based on task progress, semantic consistency, physical feasibility, and uncertainty, EA-WM improves the interpretability and alignment of world models with task objectives. The framework demonstrates significant performance improvements in various manipulation scenarios, including navigation and deformable object handling, showcasing its effectiveness in guiding planning and action selection.
Event-aware verification in world models can dramatically enhance task alignment and interpretability in robotic manipulation tasks.
Pretrained-feature world models provide a useful substrate for robot imagination, but visual or latent prediction alone does not determine whether an imagined future satisfies task-relevant events. Long-horizon manipulation requires progress signals that are relational, predicate-level, and physically grounded: whether an object has moved, whether a drawer or contact state has changed, whether a placement predicate is satisfied, and whether a candidate future is reliable enough for execution. We introduce EA-WM, an event-aware world-model framework that augments frozen visual-feature dynamics with task-specification-grounded event prediction and verification. EA-WM rolls out candidate futures in pretrained visual-feature space, decodes them into structured event states, and scores them using task-progress, semantic-consistency, physical-feasibility, and uncertainty terms. The verifier guides sampling-based planning, gates candidate actions, and, in the contact-sensitive LIBERO wine-rack setting, selects among PPOgenerated proposals. Across navigation, deformable-object, wall-constrained, and languagedescribed manipulation studies, EA-WM shows that event-aware verification can make featurespace world models more interpretable and better aligned with task progress.
