FOGMACHINE -- Leveraging Discrete-Event Simulation and Scene Graphs for Modeling Hierarchical, Interconnected Environments under Partial Observations from Mobile Agents

2025-10-10

AI Summary

Existing dynamic scene graph (DSG) approaches struggle to model embodied AI agents operating in partially observable, latency-aware, and stochastically dynamic multi-agent environments. This paper introduces the first open-source framework integrating DSGs with discrete-event simulation (DES), enabling hierarchical environment modeling, uncertainty propagation, limited-perception planning, and multi-agent collaborative reasoning. Our key contribution is the principled integration of DES into DSGs, supporting scalable, probabilistic modeling of stochastic dynamics, perceptual latency, and asynchronous agent behavior. Evaluated on urban spatiotemporal scenarios, the framework accurately reproduces real-world behavioral patterns and identifies critical bottlenecks in belief estimation under sparse observations. It provides embodied AI researchers with a scalable, interpretable simulation and evaluation platform grounded in rigorous formal semantics.

Technology Category

• Multiagent Systems: Agent-Based Simulation and Emergent Behavior
• Planning, Routing, and Scheduling: Model-Based Reasoning
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Search and Retrieval-Augmented AI: Agentic search

Abstract

Dynamic Scene Graphs (DSGs) provide a structured representation of hierarchical, interconnected environments, but current approaches struggle to capture stochastic dynamics, partial observability, and multi-agent activity. These aspects are critical for embodied AI, where agents must act under uncertainty and delayed perception. We introduce FOGMACHINE , an open-source framework that fuses DSGs with discrete-event simulation to model object dynamics, agent observations, and interactions at scale. This setup enables the study of uncertainty propagation, planning under limited perception, and emergent multi-agent behavior. Experiments in urban scenarios illustrate realistic temporal and spatial patterns while revealing the challenges of belief estimation under sparse observations. By combining structured representations with efficient simulation, FOGMACHINE establishes an effective tool for benchmarking, model training, and advancing embodied AI in complex, uncertain environments.

Problem

• Modeling stochastic dynamics in hierarchical environments under partial observability
• Studying uncertainty propagation and planning with limited perception capabilities
• Analyzing emergent multi-agent behavior in complex uncertain environments

Innovation

• Combines scene graphs with discrete-event simulation
• Models object dynamics and multi-agent interactions
• Enables uncertainty propagation and belief estimation