Lingjing: A Simulation Testbed for Multi-Agent Embodied Tasks in Open-Ended Cities

2026-08-08

AI Summary

This work addresses the lack of a unified, scalable, and closed-loop simulation evaluation platform for heterogeneous embodied agents—such as drones, ground robots, and autonomous vehicles—to collaboratively perform tasks in dynamic, open-world urban environments. The authors present a geospatial data-driven multi-agent embodied intelligence simulation framework that, for the first time, integrates dynamic urban environment reconstruction, synchronized multi-physics engines, natural language–based task specification, agent control via Gym-compatible interfaces and the ReAct framework, and systematic evaluation with support for resource constraints, communication configurations, and execution provenance tracking. Evaluated across nine urban task categories using twelve vision-language models, the platform reveals critical bottlenecks in embodied reasoning and long-horizon execution, while empirically demonstrating task-dependent cooperation trade-offs and diminishing returns from resource scaling.

Abstract

Urban embodied intelligence requires coordination among heterogeneous agents (e.g., UAVs, ground robots, and autonomous vehicles) in dynamic cities. Simulators therefore provide a scalable foundation for developing and evaluating such coordination. Existing platforms nevertheless isolate different embodiments and decouple them from task design and evaluation. We present \textbf{Lingjing}, a simulation platform for heterogeneous multi-agent embodied intelligence in open-ended urban environments. Lingjing reconstructs and renders evolving cities from geographic data, synchronizes multiple physics engines, and exposes shared physical and structured urban state to agents. Its Gym-like interface supports user-defined ReAct agents and single- or multi-agent natural-language missions with configurable star or broadcast communication and resource constraints. Each episode becomes an attribution-ready replay that links agent trajectories and communication to relation-graph changes, resource consumption, and engine-based evaluations for systematic diagnosis. We evaluate twelve vision-language models on nine urban tasks under a shared engine-in-the-loop protocol. Controlled studies further examine communication, scalability, robustness, and failure provenance. Results expose persistent bottlenecks in grounding and long-horizon execution. They also show task-dependent coordination trade-offs and diminishing returns from added capacity, while heavier workloads further reduce success. Lingjing provides a unified testbed that enables reproducible end-to-end evaluation and systematic failure diagnosis in urban multi-agent embodied intelligence.

Problem

• multi-agent embodied intelligence
• urban simulation
• heterogeneous agents
• task coordination
• open-ended cities

Innovation

• multi-agent embodied intelligence
• urban simulation
• heterogeneous agents
• physics engine synchronization
• attribution-ready replay