Mosaic: Runtime-Efficient Multi-Agent Embodied Planning

2026-07-10

AI Summary

This work addresses key challenges in large language model (LLM)-driven multi-agent embodied planning, including high execution latency, inaccurate state tracking, and inefficient coordination. To overcome these limitations, the authors propose Mosaic, a novel framework that integrates agent-centric relative-coordinate semantic memory with an integer linear programming (ILP)-based coordination mechanism. This approach enables lightweight yet precise state modeling and physically feasible action assignment. Evaluated on the AI2-THOR and search-and-rescue benchmarks, Mosaic reduces runtime overhead significantly, achieving 27–32% faster execution, 30–33% fewer LLM invocations, 25–31% fewer planning steps, and a 4–10 percentage point improvement in task success rate compared to existing methods.

Abstract

LLM-based multi-agent embodied planning remains impractical due to prohibitively high execution latency. We identify failed actions as the dominant bottleneck, stemming from two core challenges: inaccurate state tracking under partial observability and inefficient coordination that produces redundant or conflicting actions. We introduce Mosaic, a runtime-efficient multi-agent planning framework that addresses both challenges. Mosaic maintains accurate yet lightweight state tracking through agent-centric semantic memory that stores objects in relative coordinates, enabling geometric transformations and coordination. It ensures efficient coordination through Integer Linear Programming that allocates actions at every planning step, enforcing physical feasibility and inter-agent coordination constraints. Across AI2-THOR and search-and-rescue benchmarks, Mosaic achieves 27-32% faster execution, 30-33% fewer LLM calls, 25-31% fewer steps, and 4-10% points higher success rates. These results demonstrate that efficient memory and constraint-guided coordination are critical for scalable, low-latency multi-agent planning.

Problem

• multi-agent planning
• execution latency
• failed actions
• state tracking
• agent coordination

Innovation

• multi-agent planning
• semantic memory
• integer linear programming
• embodied AI
• runtime efficiency