Functional 3D Scene Synthesis through Human-Scene Optimization

2025-02-05

AI Summary

This work addresses critical challenges in text-driven 3D indoor scene generation—namely, object occlusion, structural rigidity, and functional implausibility—by introducing a human-centric functional interaction constraint. We propose the first generative framework incorporating human-object co-optimization, wherein a graph diffusion network synthesizes semantically coherent scene graphs, augmented by functional co-occurrence modeling and 3D human-object interaction reasoning to jointly optimize physical feasibility, semantic consistency, and natural spatial layout. Experiments demonstrate substantial improvements in functional coherence and spatial plausibility: collision rate decreases by 42%, interaction feasibility increases by 37%, and our method outperforms state-of-the-art approaches on both quantitative metrics and qualitative evaluation.

Technology Category

• Humans and AI: Interaction Techniques and Devices
• Computer Vision: Diffusion Models for Vision
• Constraint Satisfaction and Optimization: Satisfiability Modulo Theories

Application Category

• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

This paper presents a novel generative approach that outputs 3D indoor environments solely from a textual description of the scene. Current methods often treat scene synthesis as a mere layout prediction task, leading to rooms with overlapping objects or overly structured scenes, with limited consideration of the practical usability of the generated environment. Instead, our approach is based on a simple, but effective principle: we condition scene synthesis to generate rooms that are usable by humans. This principle is implemented by synthesizing 3D humans that interact with the objects composing the scene. If this human-centric scene generation is viable, the room layout is functional and it leads to a more coherent 3D structure. To this end, we propose a novel method for functional 3D scene synthesis, which consists of reasoning, 3D assembling and optimization. We regard text guided 3D synthesis as a reasoning process by generating a scene graph via a graph diffusion network. Considering object functional co-occurrence, a new strategy is designed to better accommodate human-object interaction and avoidance, achieving human-aware 3D scene optimization. We conduct both qualitative and quantitative experiments to validate the effectiveness of our method in generating coherent 3D scene synthesis results.

Problem

• Generates 3D indoor environments from text
• Ensures human usability in scene synthesis
• Optimizes for human-object interaction and avoidance

Innovation

• Text-guided 3D scene synthesis
• Human-centric scene optimization
• Graph diffusion network reasoning