Imperative vs. Declarative Programming Paradigms for Open-Universe Scene Generation

2025-04-07

AI Summary

This paper addresses open-vocabulary, text-driven 3D scene layout generation—i.e., placing a set of objects in 3D space according to natural language descriptions. To overcome the poor generalizability and limited semantic expressiveness of existing declarative constraint-solving approaches, we propose the first LLM-based progressive *imperative* layout generation paradigm. Our method iteratively generates object placement instructions, models geometric constraints, and incorporates error feedback for correction—preserving the LLM’s semantic intent while enhancing layout validity. We further introduce a perception-consistent automatic evaluation framework aligned with human preferences. Experiments show that our approach achieves 82% and 94% preference rates over two leading declarative methods in forced-choice perceptual tests, significantly improving generation success rate and plausibility for complex, open-ended scenes.

Technology Category

• Planning, Routing, and Scheduling: Planning with Language Models
• Natural Language Processing: Generation
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information

Abstract

Synthesizing 3D scenes from open-vocabulary text descriptions is a challenging, important, and recently-popular application. One of its critical subproblems is layout generation: given a set of objects, lay them out to produce a scene matching the input description. Nearly all recent work adopts a declarative paradigm for this problem: using LLM to generate specification of constraints between objects, then solving those constraints to produce the final layout. In contrast, we explore an alternative imperative paradigm, in which an LLM iteratively places objects, with each object's position and orientation computed as a function of previously-placed objects. The imperative approach allows for a simpler scene specification language while also handling a wider variety and larger complexity of scenes. We further improve the robustness of our imperative scheme by developing an error correction mechanism that iteratively improves the scene's validity while staying as close as possible the original layout generated by the LLM. In forced-choice perceptual studies, participants preferred layouts generated by our imperative approach 82% and 94% of the time, respectively, when compared against two declarative layout generation methods. We also present a simple, automated evaluation metric for 3D scene layout generation that aligns well with human preferences.

Problem

• Comparing imperative vs declarative paradigms for 3D scene generation
• Improving layout robustness via iterative error correction
• Evaluating human preference alignment in generated layouts

Innovation

• LLM iteratively places objects in scenes
• Error correction improves scene validity
• Imperative paradigm simplifies scene specification