Explore Like Humans: Autonomous Exploration with Online SG-Memo Construction for Embodied Agents

2026-04-20

AI Summary

This work addresses the limitations of existing embodied agents, which typically rely on post-hoc, geometry-centric offline methods for spatial memory construction, often failing to integrate high-level semantics and overlooking critical navigational landmarks. To overcome this, the authors propose ABot-Explorer, a novel framework that achieves the first online coupling of exploration and structured semantic graph memory (SG-Memo). Driven solely by RGB input, the agent leverages a large vision-language model to extract semantic navigation affordances (SNAs) as cognitive anchors, guiding it to prioritize structurally salient transition nodes during exploration. The approach introduces a cognitively aligned SNA mechanism and contributes the first large-scale dataset annotated with both SNAs and SG-Memo. Experiments demonstrate significant improvements over state-of-the-art methods in exploration efficiency and environmental coverage, with the constructed SG-Memo effectively supporting diverse downstream tasks.

Technology Category

• Intelligent Robots: Embodied AI
• Cognitive Modeling & Cognitive Systems: Agent Architectures
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Agentic search
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Constructing structured spatial memory is essential for enabling long-horizon reasoning in complex embodied navigation tasks. Current memory construction predominantly relies on a decoupled, two-stage paradigm: agents first aggregate environmental data through exploration, followed by the offline reconstruction of spatial memory. However, this post-hoc and geometry-centric approach precludes agents from leveraging high-level semantic intelligence, often causing them to overlook navigationally critical landmarks (e.g., doorways and staircases) that serve as fundamental semantic anchors in human cognitive maps. To bridge this gap, we propose ABot-Explorer, a novel active exploration framework that unifies memory construction and exploration into an online, RGB-only process. At its core, ABot-Explorer leverages Large Vision-Language Models (VLMs) to distill Semantic Navigational Affordances (SNA), which act as cognitive-aligned anchors to guide the agent's movement. By dynamically integrating these SNAs into a hierarchical SG-Memo, ABot-Explorer mirrors human-like exploratory logic by prioritizing structural transit nodes to facilitate efficient coverage. To support this framework, we contribute a large-scale dataset extending InteriorGS with SNA and SG-Memo annotations. Experimental results demonstrate that ABot-Explorer significantly outperforms current state-of-the-art methods in both exploration efficiency and environment coverage, while the resulting SG-Memo is shown to effectively support diverse downstream tasks.

Problem

• spatial memory
• embodied navigation
• semantic landmarks
• autonomous exploration
• long-horizon reasoning

Innovation

• Semantic Navigational Affordances
• Online Memory Construction
• Large Vision-Language Models
• Embodied Exploration
• SG-Memo