SimWorlds: A Multi-Agent System for Dynamic 3D Scene Creation

2026-07-02

AI Summary

Existing text-to-3D scene generation methods produce only static outputs and struggle to model physical dynamic behaviors such as fluids, particles, and rigid body motion. This work proposes the first multi-agent framework capable of generating editable, physically consistent dynamic 4D scenes directly from text. The approach employs a planner–encoder–reviewer collaborative workflow to construct spatiotemporally coherent dynamic scenes in Blender through staged refinement, augmented with runtime state monitoring and deterministic verification mechanisms to ensure physical correctness. Additionally, we introduce a hierarchical scene protocol and establish 4DBuildBench, the first benchmark for evaluating dynamic 4D scene generation. Experiments demonstrate that our method significantly outperforms existing dynamic Blender generation baselines in both visual fidelity and physical consistency.

Abstract

LLM agents are increasingly used to translate natural language into 3D scenes in a procedural way, but existing systems focus on static output. Dynamic 4D scenes from text alone, in which liquids flow, particles emit, rigid bodies cascade, and articulated mechanisms move, remain largely unexplored despite their value as editable content and as physics-grounded training data for video generation and embodied AI. Two challenges set the dynamic case apart from static text-to-scene work: an agent must jointly coordinate spatial layout, multiple physics solvers, temporal sequencing, camera, and lighting in a single coherent scene, and verifying motion correctness from rendered video is fundamentally harder than judging a single image. We present SimWorlds: a multi-agent framework that produces dynamic, editable 4D scenes from text, with Blender-specific procedural knowledge, a planner-coder-reviewer workflow driving a fixed ordered sequence of construction stages, a layered scene protocol enforced by a deterministic verifier, and a runtime-state inspection tool suite that catches mechanism failures the rendered image cannot reveal. We also introduce 4DBuildBench, a benchmark for assessing both visual fidelity and physical consistency of the procedural dynamic 3D scenes generated from text prompts. Experiments show that SimWorlds outperforms prior dynamic Blender generation baselines.

Problem

• dynamic 4D scenes
• text-to-scene generation
• physics-based animation
• multi-agent system
• procedural content creation

Innovation

• multi-agent system
• dynamic 4D scene generation
• physics-grounded simulation
• procedural 3D content
• verifiable scene construction