Flame3D: Zero-shot Compositional Reasoning of 3D Scenes with Agentic Language Models

2026-05-09

AI Summary

Existing 3D scene understanding approaches either rely on large-scale 3D-language pretraining or support only simple spatial relations, limiting their capacity for complex, open-ended compositional reasoning. This work proposes Flame3D, a zero-shot framework that requires no 3D-specific training. It encodes scenes into an editable visuo-linguistic memory and leverages off-the-shelf multimodal large language models in conjunction with composable spatial tools to dynamically synthesize custom spatial programs for reasoning about object layouts, free space, and unseen objects. Flame3D matches the performance of fine-tuned 3D language models on ScanQA and demonstrates—on the Compose3D benchmark—the critical role of runtime program synthesis in enabling multi-hop reasoning, achieving, for the first time, sophisticated compositional 3D reasoning without any task-specific training.

Abstract

3D scene understanding spans reasoning about free space, object grounding, hypothetical object insertions, complex geometric relationships, and integrating all of these with external tools and data sources. Existing 3D understanding methods typically rely on large-scale 3D-language training or focus on object grounding and simple spatial relationships. We argue that the broad generalization that motivates 3D-language training can be achieved at inference time, without 3D-specific training. We propose Flame3D, a training-free framework that represents scenes as editable visual-textual 3D memories and exposes them to an off-the-shelf MLLM through composable spatial tools. Flame3D also lets the agent synthesize custom spatial programs at inference time, enabling open-ended reasoning over layouts, empty space, and objects not yet present in the scene. External data and corrections can be added to the memory without retraining. In addition to showing competitive performance to finetuned 3D-LMM methods on ScanQA, we study multi-hop 3D reasoning capabilities of Flame3D by evaluating it on a curated compositional spatial-reasoning benchmark, Compose3D. We find that fixed tools fall short and that the agent's ability to synthesize spatial operations at inference time is essential. These results invite the question: should future progress in 3D scene understanding focus on richer scene memories and expressive compositional abstractions?

Problem

• 3D scene understanding
• compositional reasoning
• zero-shot
• spatial relationships
• object grounding

Innovation

• zero-shot
• compositional reasoning
• 3D scene understanding
• agentic language models
• spatial program synthesis