E2Map: Experience-and-Emotion Map for Self-Reflective Robot Navigation with Language Models

2024-09-16

arXiv.org

AI Summary

To address the poor adaptability of LLM-driven robotic navigation in dynamic and uncertain environments—and its inability to leverage embodied experience or interact meaningfully with uncertainty—this paper proposes the Experience-and-Emotion Map (E2Map): an online, incrementally updatable spatial representation integrating LLM-based priors, embodied experience memory, and emotion-inspired uncertainty assessment. It is the first work to incorporate embodied experience modeling and emotion-informed heuristic feedback into LLM-based navigation, enabling self-reflective, single-step behavioral correction and relaxing the common static-environment assumption. The method comprises four components: LLM-based reasoning, experience encoding, dynamic map updating, and joint simulation–real-robot validation. Evaluated across diverse stochastic navigation tasks, E2Map significantly improves task success rate and robustness over state-of-the-art LLM-based baselines.

Technology Category

• Intelligent Robots: Embodied AI
• Natural Language Processing: Safety and Robustness
• Planning, Routing, and Scheduling: Planning with Language Models

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Large language models (LLMs) have shown significant potential in guiding embodied agents to execute language instructions across a range of tasks, including robotic manipulation and navigation. However, existing methods are primarily designed for static environments and do not leverage the agent's own experiences to refine its initial plans. Given that real-world environments are inherently stochastic, initial plans based solely on LLMs' general knowledge may fail to achieve their objectives, unlike in static scenarios. To address this limitation, this study introduces the Experience-and-Emotion Map (E2Map), which integrates not only LLM knowledge but also the agent's real-world experiences, drawing inspiration from human emotional responses. The proposed methodology enables one-shot behavior adjustments by updating the E2Map based on the agent's experiences. Our evaluation in stochastic navigation environments, including both simulations and real-world scenarios, demonstrates that the proposed method significantly enhances performance in stochastic environments compared to existing LLM-based approaches. Code and supplementary materials are available at https://e2map.github.io/.

Problem

• Adaptability
• Uncertainty Handling
• Language Understanding

Innovation

• E2Map
• Large Language Models Integration
• Adaptive Behavior in Uncertain Environments