Track manipulation, locomotion and embodied systems.
Search papers, labs, and topics across Lattice.
This paper introduces ManiUnit, a novel dataset and benchmark designed to address the challenges of learning and evaluating long-horizon mobile manipulation skills. ManiUnit comprises 137,899 skill segments across 21 types and 417 subtasks, featuring explicit subtask instructions and enabling isolated skill evaluation by restoring intermediate simulator states and defining local success conditions. Evaluations reveal that current vision-language-action policies exhibit significant per-skill performance disparities, and that joint perturbations can degrade execution by approximately 56%, highlighting the importance of robust skill learning for long-horizon task completion.
Robot skills degrade significantly (56% success drop) from minor starting state perturbations, yet isolated skill evaluation reveals substantial per-skill performance gaps hidden by aggregate metrics.
Long-horizon mobile manipulation requires a robot to navigate multi-room environments and execute a sequence of manipulation skills under a single natural language instruction. Learning and evaluating these skills present three challenges: similar observations under a fixed task instruction may make skill selection ambiguous; even when a preceding skill succeeds, the robot state inherited by the next skill may deviate from its demonstrated starting states and affect execution; and task-level metrics hinder skill-specific diagnosis, while early failures leave later skills untested. We therefore introduce ManiUnit, a manipulation skill dataset and benchmark built from 50 BEHAVIOR-1K activities. Its dataset contains 137,899 segments across 21 skill types and 417 subtasks, and its benchmark contains 1,260 test instances. Correspondingly, ManiUnit pairs each segment with an explicit subtask instruction; measures sensitivity to perturbations of the robot's starting base position or joint configuration; and restores intermediate simulator states and defines local success conditions so that each skill can be evaluated without executing preceding stages. Evaluations of representative vision-language-action (VLA) policies show that similar aggregate scores can hide substantial per-skill differences. The tested starting-state perturbations also degrade execution: on the full benchmark, joint perturbations reduce success rates by approximately 56% relative to those from demonstrated starting states. On two long-horizon activities, a skill policy trained on ManiUnit segments achieves 78.7% local manipulation success, compared with 49.3% for a task policy trained on complete demonstrations. The trained skills further support complete-task execution on these activities, as coordinating the task and skill policies through a planner raises full-task success from 4.0% to 18.0%.