🤖 AI Summary
Deploying large language models (LLMs) for embodied humanoid robots faces four key challenges: safety assurance, seamless cross-task transition, multi-timescale task adaptation, and closed-loop state feedback.
Method: We propose a GPT-4–empowered hierarchical control framework integrating prompt engineering, real-time sensor feedback, hierarchical behavior state machines, and dynamic re-planning to enable reliable, goal-directed subtask decomposition and execution in both simulation and real-world settings.
Contribution/Results: This work is the first to systematically address safety-constrained LLM deployment for embodied agents, ensure task-transition consistency, and close the perception–decision–action loop. Experiments demonstrate 100% executable subtask generation across all timescales, jitter-free task switching, and significantly higher user-goal success rates versus baseline methods—substantially enhancing the practical deployability of LLM-driven robots in real-world scenarios.
📝 Abstract
We explore the use of GPT-4 on a humanoid robot in simulation and the real world as proof of concept of a novel large language model (LLM) driven behaviour method. LLMs have shown the ability to perform various tasks, including robotic agent behaviour. The problem involves prompting the LLM with a goal, and the LLM outputs the sub-tasks to complete to achieve that goal. Previous works focus on the executability and correctness of the LLM's generated tasks. We propose a method that successfully addresses practical concerns around safety, transitions between tasks, time horizons of tasks and state feedback. In our experiments we have found that our approach produces output for feasible requests that can be executed every time, with smooth transitions. User requests are achieved most of the time across a range of goal time horizons.