Scaling Manual-Grounded Appliance Manipulation with Data Synthesis and Unified Planning

2026-08-16

AI Summary

This study addresses the scarcity of appliance manipulation data and the limited long-horizon planning capabilities of large language models by proposing the MAGE data synthesis pipeline, the UseAppliance dataset, and an end-to-end model named AppliancePlan. This approach introduces a novel hierarchical appliance graph auto-generation technique that enables manual-based operational planning with closed-loop recovery. Experimental results demonstrate that the 7B-parameter model achieves open-loop planning performance ten times superior to state-of-the-art baselines, leads across all task metrics, and effectively transfers to real-world scenarios involving six appliance categories. By bridging the gap in large-scale, manual-driven manipulation datasets, this work significantly advances the development of general-purpose domestic robots capable of robust household task execution.

Abstract

Operating household appliances requires long-horizon planning that is state-dependent and robust to disturbances, yet existing large models fall short, as no sufficiently diverse, task-oriented dataset exists to support such planning. To bridge this gap, we propose MAGE, a scalable data synthesis pipeline that introduces a novel Hierarchical Appliance Graph (HAG) to automatically generate part grounding, long-horizon planning, and closed-loop recovery data from appliance manuals. With MAGE, we build UseAppliance, the first large-scale dataset for manual-grounded appliance manipulation planning, spanning 22 appliance categories with 89K+ part annotations, 53K+ manipulation tasks, and 33K+ closed-loop adjustment steps. Built on UseAppliance, we develop AppliancePlan, an end-to-end model for manual-grounded appliance manipulation planning. On RealAppliance-Bench, AppliancePlan with only 7B parameters achieves over 10x the best baseline on open-loop planning and consistently outperforms state-of-the-art models across all tasks. Real-robot experiments on six household appliances further confirm effective sim-to-real transfer, marking an important step toward general-purpose household robotics.

Problem

• Appliance Manipulation
• Long-horizon Planning
• Data Scarcity
• Manual-Grounded

Innovation

• Hierarchical Appliance Graph
• Data Synthesis
• Manual-Grounded Planning
• UseAppliance Dataset
• Sim-to-Real Transfer