NaLA: A 3D Native LLM Layout Agent for High-quality 3D Scene Generation

2026-06-28

AI Summary

Existing large language model–based approaches to 3D scene layout rely on textual descriptions, suffering from a modality gap that leads to loss of geometric detail and implausible arrangements. This work proposes NaLA, the first natively 3D-input/output large language model layout agent, which directly embeds 3D assets and combines autoregressive discrete pose prediction with continuous pose regression for refinement, enabling high-quality layout generation. By preserving fine-grained geometric information and explicitly reasoning about spatial relationships—such as collisions, support, and containment—NaLA achieves superior generation quality and inference efficiency compared to existing methods. Ablation studies confirm the effectiveness of each component, demonstrating a significant improvement in geometric awareness and layout consistency.

Abstract

Recently, Large Language Models (LLMs) have emerged as promising layout agents for 3D scene generation. Existing layout agents still suffer from implausible layout generation because most of them convert 3D assets and 3D layouts into textual descriptions as inputs and outputs, which involves severe information loss due to the modality gap between texts and 3D assets and 3D layouts. We propose NaLA, a native 3D LLM layout Agent for high-quality 3D scene generation by placing 3D assets in the scene. For the inputs, NaLA encodes 3D scene boundaries and 3D assets directly into the LLM, preserving fine-grained geometry and enabling explicit reasoning over relationships like collisions, surface supporting, and containment. To accurately output the positions and orientations of assets, NaLA adopts a coarse-to-fine prediction mechanism that first predicts discrete poses in an autoregressive manner and then refines the discrete poses with a continuous regression. Trained on diverse layout datasets, NaLA attains strong geometric perception and layout coherence. Experiments demonstrate that NaLA outperforms prior layout agents in both generation quality and inference efficiency, with comprehensive ablation studies to verify each component's effectiveness.

Problem

• 3D scene generation
• layout agent
• modality gap
• geometric perception
• LLM

Innovation

• 3D Native LLM
• Layout Generation
• Geometric Reasoning
• Coarse-to-Fine Prediction
• Modality Gap