Towards Embodied Agentic AI: Review and Classification of LLM- and VLM-Driven Robot Autonomy and Interaction

2025-08-07

AI Summary

Enhancing robotic autonomy and interactive capability remains challenging due to fragmented integration of foundational AI models. Method: This work systematically investigates the synergistic deployment of large language models (LLMs), vision-language models (VLMs), vision-language-action models (VLAs), and behavior-large models (BLMs) in embodied agents. We propose a novel agent role taxonomy that unifies functional modeling across natural language understanding, task sequencing, API invocation, action execution, and diagnostic assistance. Integrating ROS and other robotic middleware, we construct the first holistic agent-driven robotics analysis framework spanning model capabilities, system architecture, and open-source ecosystem. Contribution/Results: Through rigorous cross-method benchmarking, we elucidate evolutionary trends and practical bottlenecks in embodied intelligence. The framework provides structured theoretical foundations and engineering guidelines for designing, evaluating, and deploying embodied agents—bridging gaps between AI model advances and real-world robotic systems.

Technology Category

• Intelligent Robots: Embodied AI
• Cognitive Modeling & Cognitive Systems: Agent Architectures
• Multiagent Systems: Agent/AI Theories and Architectures

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Agentic search
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Foundation models, including large language models (LLMs) and vision-language models (VLMs), have recently enabled novel approaches to robot autonomy and human-robot interfaces. In parallel, vision-language-action models (VLAs) or large behavior models (BLMs) are increasing the dexterity and capabilities of robotic systems. This survey paper focuses on those words advancing towards agentic applications and architectures. This includes initial efforts exploring GPT-style interfaces to tooling, as well as more complex system where AI agents are coordinators, planners, perception actors, or generalist interfaces. Such agentic architectures allow robots to reason over natural language instructions, invoke APIs, plan task sequences, or assist in operations and diagnostics. In addition to peer-reviewed research, due to the fast-evolving nature of the field, we highlight and include community-driven projects, ROS packages, and industrial frameworks that show emerging trends. We propose a taxonomy for classifying model integration approaches and present a comparative analysis of the role that agents play in different solutions in today's literature.

Problem

• Advancing robot autonomy using LLMs and VLMs
• Enhancing human-robot interaction through agentic AI
• Classifying model integration approaches in robotics

Innovation

• LLM and VLM enhance robot autonomy
• VLA and BLM boost robotic dexterity
• Agentic architectures enable language reasoning