Semantic Area Graph Reasoning for Multi-Robot Language-Guided Search

2026-04-17

AI Summary

This work addresses the challenge of efficiently integrating high-level natural language instructions—such as “find a cup in the kitchen”—with geometric exploration in multi-robot systems operating in unknown environments. To this end, the paper proposes the Semantic Area Graph Reasoning (SAGR) framework, which introduces, for the first time, a structured semantic area graph as an interface between large language models and multi-robot coordination. The approach constructs a semantic topological abstraction of the environment via semantic occupancy mapping, leverages a large language model for high-level semantic room assignment, and integrates deterministic frontier-based planning for local navigation. Evaluated across 100 scenes from the Habitat-Matterport3D dataset, the method achieves up to an 18.8% improvement in semantic target search efficiency while maintaining state-of-the-art exploration performance.

Technology Category

• Intelligent Robots: Multi-Robot Systems
• Planning, Routing, and Scheduling: Planning with Language Models
• Multiagent Systems: Multiagent Planning

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Coordinating multi-robot systems (MRS) to search in unknown environments is particularly challenging for tasks that require semantic reasoning beyond geometric exploration. Classical coordination strategies rely on frontier coverage or information gain and cannot incorporate high-level task intent, such as searching for objects associated with specific room types. We propose \textit{Semantic Area Graph Reasoning} (SAGR), a hierarchical framework that enables Large Language Models (LLMs) to coordinate multi-robot exploration and semantic search through a structured semantic-topological abstraction of the environment. SAGR incrementally constructs a semantic area graph from a semantic occupancy map, encoding room instances, connectivity, frontier availability, and robot states into a compact task-relevant representation for LLM reasoning. The LLM performs high-level semantic room assignment based on spatial structure and task context, while deterministic frontier planning and local navigation handle geometric execution within assigned rooms. Experiments on the Habitat-Matterport3D dataset across 100 scenarios show that SAGR remains competitive with state-of-the-art exploration methods while consistently improving semantic target search efficiency, with up to 18.8\% in large environments. These results highlight the value of structured semantic abstractions as an effective interface between LLM-based reasoning and multi-robot coordination in complex indoor environments.

Problem

• multi-robot coordination
• semantic search
• unknown environments
• language-guided tasks
• semantic reasoning

Innovation

• Semantic Area Graph
• Large Language Models
• Multi-Robot Coordination
• Semantic Exploration
• Hierarchical Reasoning