Closed-Loop Long-Horizon Robotic Planning via Equilibrium Sequence Modeling

2024-10-02

arXiv.org

AI Summary

Long-horizon task planning for autonomous robots faces challenges in reliably translating high-level instructions into executable action sequences, while existing language-model-based agents suffer from limited foresight and error-proneness. Method: This paper proposes a closed-loop self-correcting planning framework grounded in balanced sequence modeling. Contribution/Results: Its core innovations are (1) the first end-to-end differentiable self-correction mechanism—requiring no external verifiers or reward models—and (2) a nested balanced sequence modeling architecture that integrates environmental feedback for efficient closed-loop iterative optimization. Trained via supervised end-to-end learning, the framework achieves significant improvements in long-horizon planning accuracy and reasoning scalability on the VirtualHome-Env benchmark, outperforming all state-of-the-art language-model agents across key metrics.

Technology Category

• Planning, Routing, and Scheduling: Planning with Language Models
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Multiagent Systems: Multiagent Planning

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

In the endeavor to make autonomous robots take actions, task planning is a major challenge that requires translating high-level task descriptions to long-horizon action sequences. Despite recent advances in language model agents, they remain prone to planning errors and limited in their ability to plan ahead. To address these limitations in robotic planning, we advocate a self-refining scheme that iteratively refines a draft plan until an equilibrium is reached. Remarkably, this process can be optimized end-to-end from an analytical perspective without the need to curate additional verifiers or reward models, allowing us to train self-refining planners in a simple supervised learning fashion. Meanwhile, a nested equilibrium sequence modeling procedure is devised for efficient closed-loop planning that incorporates useful feedback from the environment (or an internal world model). Our method is evaluated on the VirtualHome-Env benchmark, showing advanced performance with improved scaling w.r.t. inference-time computation. Code is available at https://github.com/Singularity0104/equilibrium-planner.

Problem

• Addressing long-horizon robotic task planning errors
• Enhancing autonomous robot action sequence planning
• Improving closed-loop planning with environmental feedback

Innovation

• Self-refining scheme for iterative plan refinement
• End-to-end optimization without additional verifiers
• Nested equilibrium modeling for closed-loop planning