4DSynth: Controllable Procedural World Synthesis for Dynamic Embodied Simulation

2026-08-27

AI Summary

本文提出4DSynth系统，通过自然语言、蓝图或照片生成可控的4D环境，解决动态实体模拟中环境多样性与可编辑性问题。

Abstract

Embodied agents need environments that are visually diverse, physically interactive, and changing over time. Procedural simulators can generate large interactive scene collections, and recent 4D generators produce compelling visual dynamics. Combining these properties in one environment, however, still demands extensive manual effort, and the result is rarely editable or controllable enough to reuse at scale. We present 4DSynth, a controllable procedural system that turns a natural-language description, a blueprint mask, or a single photograph into an editable 4D environment with explicit geometry, animated actors, collision-free trajectories, and physics-ready simulation state. Multiple scene routes share one geometry-grounded representation, so the same pipeline handles animation, camera planning, rendering, and task generation. To validate the full pipeline, we construct 4DSynth-Nav, an interactive navigation benchmark generated entirely from 4DSynth's procedural scenes. Two vision-language models evaluated across three difficulty tiers both fail the majority of tasks and stall after early subtasks. The same procedural controllability that produces these environments also makes each failure reproducible and each difficulty axis independently tunable. This paper presents both a controllable generation pipeline and the scalable benchmark it enables, offering a practical foundation for developing and evaluating embodied agents.

Problem

• procedural simulation
• visual diversity
• physical interaction
• time dynamics
• controllable

Innovation

• Controllable Procedural System
• 4D Environment
• Natural-Language Description
• Editable and Interactive
• Scalable Benchmark