Search papers, labs, and topics across Lattice.
This paper introduces MAGE, a data synthesis pipeline that leverages a Hierarchical Appliance Graph (HAG) to generate diverse datasets for manual-grounded appliance manipulation, addressing the lack of suitable data for long-horizon planning in household robotics. The resulting UseAppliance dataset features over 89K part annotations and 53K manipulation tasks across 22 appliance categories, enabling the development of AppliancePlan, an end-to-end model that excels in planning tasks. AppliancePlan, with only 7B parameters, significantly outperforms existing models on the RealAppliance-Bench and demonstrates effective sim-to-real transfer in real-robot experiments, paving the way for advancements in general-purpose household robotics.
A novel data synthesis approach enables a lightweight model to outperform state-of-the-art planning systems by over 10x in household appliance manipulation tasks.
Operating household appliances requires long-horizon planning that is state-dependent and robust to disturbances, yet existing large models fall short, as no sufficiently diverse, task-oriented dataset exists to support such planning. To bridge this gap, we propose MAGE, a scalable data synthesis pipeline that introduces a novel Hierarchical Appliance Graph (HAG) to automatically generate part grounding, long-horizon planning, and closed-loop recovery data from appliance manuals. With MAGE, we build UseAppliance, the first large-scale dataset for manual-grounded appliance manipulation planning, spanning 22 appliance categories with 89K+ part annotations, 53K+ manipulation tasks, and 33K+ closed-loop adjustment steps. Built on UseAppliance, we develop AppliancePlan, an end-to-end model for manual-grounded appliance manipulation planning. On RealAppliance-Bench, AppliancePlan with only 7B parameters achieves over 10x the best baseline on open-loop planning and consistently outperforms state-of-the-art models across all tasks. Real-robot experiments on six household appliances further confirm effective sim-to-real transfer, marking an important step toward general-purpose household robotics.
