IndoorWorld: Integrating Physical Task Solving and Social Simulation in A Heterogeneous Multi-Agent Environment

2025-06-14

AI Summary

Existing virtual environments for AI agent research treat physical task solving and social simulation as disjoint domains: the former neglects social dynamics, while the latter lacks physical grounding. This work introduces the first heterogeneous multi-agent virtual environment that unifies physical manipulation and social interaction, enabling LLM-driven agents to collaboratively execute embodied physical actions and socially situated behaviors within realistic spatial settings. Our core contributions are: (1) bidirectional dynamic anchoring between physical environment states and social behaviors; (2) a scalable, occupant-level simulation paradigm tailored for architectural human factors design; and (3) a unified framework integrating a spatially aware world model, resource competition modeling, a social behavior state machine, and a physics-based action engine. Experiments in office scenarios demonstrate significant effects of spatial layout, collaboration mechanisms, and resource constraints on collective behavior—establishing a new benchmark for AI-driven building performance simulation.

Technology Category

• Multiagent Systems: Agent-Based Simulation and Emergent Behavior
• Cognitive Modeling & Cognitive Systems: Agent Architectures
• Humans and AI: Game Design — Virtual Humans, NPCs and Autonomous Characters

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Architectures and workflows that use LLMs for crowd work
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems

Abstract

Virtual environments are essential to AI agent research. Existing environments for LLM agent research typically focus on either physical task solving or social simulation, with the former oversimplifying agent individuality and social dynamics, and the latter lacking physical grounding of social behaviors. We introduce IndoorWorld, a heterogeneous multi-agent environment that tightly integrates physical and social dynamics. By introducing novel challenges for LLM-driven agents in orchestrating social dynamics to influence physical environments and anchoring social interactions within world states, IndoorWorld opens up possibilities of LLM-based building occupant simulation for architectural design. We demonstrate the potential with a series of experiments within an office setting to examine the impact of multi-agent collaboration, resource competition, and spatial layout on agent behavior.

Problem

• Integrating physical and social dynamics in AI agents
• Addressing oversimplified agent individuality in existing environments
• Anchoring social interactions within physical world states

Innovation

• Integrates physical and social dynamics
• Anchors social interactions in world states
• Simulates multi-agent collaboration and competition