EmbodiedLGR: Integrating Lightweight Graph Representation and Retrieval for Semantic-Spatial Memory in Robotic Agents

2026-04-20

AI Summary

Existing robotic systems struggle to efficiently construct and retrieve semantic-spatial memory in complex environments, limiting the real-time performance and accuracy of human-robot interaction. This work proposes EmbodiedLGR-Agent, a novel architecture that integrates a lightweight semantic graph with a retrieval-augmented mechanism. It leverages a parameter-efficient vision-language model to build dense semantic graphs and combines low-level object-location encodings with high-level scene descriptions to form a hybrid memory representation. The approach achieves significantly improved efficiency in both memory construction and querying while maintaining high accuracy, attaining state-of-the-art performance in inference and retrieval speed on the NaVQA dataset. Furthermore, it has been successfully deployed on a physical robot, demonstrating its practicality and scalability in real-world human-robot interaction scenarios.

Technology Category

• Intelligent Robots: Embodied AI
• Cognitive Modeling & Cognitive Systems: Agent Architectures
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Agentic search
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

As the world of agentic artificial intelligence applied to robotics evolves, the need for agents capable of building and retrieving memories and observations efficiently is increasing. Robots operating in complex environments must build memory structures to enable useful human-robot interactions by leveraging the mnemonic representation of the current operating context. People interacting with robots may expect the embodied agent to provide information about locations, events, or objects, which requires the agent to provide precise answers within human-like inference times to be perceived as responsive. We propose the Embodied Light Graph Retrieval Agent (EmbodiedLGR-Agent), a visual-language model (VLM)-driven agent architecture that constructs dense and efficient representations of robot operating environments. EmbodiedLGR-Agent directly addresses the need for an efficient memory representation of the environment by providing a hybrid building-retrieval approach built on parameter-efficient VLMs that store low-level information about objects and their positions in a semantic graph, while retaining high-level descriptions of the observed scenes with a traditional retrieval-augmented architecture. EmbodiedLGR-Agent is evaluated on the popular NaVQA dataset, achieving state-of-the-art performance in inference and querying times for embodied agents, while retaining competitive accuracy on the global task relative to the current state-of-the-art approaches. Moreover, EmbodiedLGR-Agent was successfully deployed on a physical robot, showing practical utility in real-world contexts through human-robot interaction, while running the visual-language model and the building-retrieval pipeline locally.

Problem

• semantic-spatial memory
• robotic agents
• memory retrieval
• human-robot interaction
• embodied AI

Innovation

• Lightweight Graph Representation
• Retrieval-Augmented Architecture
• Visual-Language Model (VLM)
• Semantic-Spatial Memory
• Embodied AI