CasaGPT: Cuboid Arrangement and Scene Assembly for Interior Design

2025-04-28

AI Summary

This work addresses severe object interpenetration and physically implausible configurations in indoor 3D scene synthesis. We propose an autoregressive generative framework based on oriented cuboid primitives—departing from conventional axis-aligned bounding box representations by parameterizing object geometry and pose via rotatable cuboids. To explicitly enforce physical plausibility, we introduce a collision-aware rejection sampling fine-tuning strategy. Furthermore, we construct a denoised dataset, 3DFRONT-NC, and design targeted data augmentation to mitigate annotation noise. Evaluated on both 3D-FRONT and 3DFRONT-NC, our method achieves new state-of-the-art performance: it significantly reduces scene interpenetration rates while markedly improving physical validity and visual realism. Ablations confirm the effectiveness and generalizability of cuboid primitives for structured 3D scene generation.

Technology Category

• Computer Vision: 3D Computer Vision
• Search and Optimization: Sampling/Simulation-based Search
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems

Abstract

We present a novel approach for indoor scene synthesis, which learns to arrange decomposed cuboid primitives to represent 3D objects within a scene. Unlike conventional methods that use bounding boxes to determine the placement and scale of 3D objects, our approach leverages cuboids as a straightforward yet highly effective alternative for modeling objects. This allows for compact scene generation while minimizing object intersections. Our approach, coined CasaGPT for Cuboid Arrangement and Scene Assembly, employs an autoregressive model to sequentially arrange cuboids, producing physically plausible scenes. By applying rejection sampling during the fine-tuning stage to filter out scenes with object collisions, our model further reduces intersections and enhances scene quality. Additionally, we introduce a refined dataset, 3DFRONT-NC, which eliminates significant noise presented in the original dataset, 3D-FRONT. Extensive experiments on the 3D-FRONT dataset as well as our dataset demonstrate that our approach consistently outperforms the state-of-the-art methods, enhancing the realism of generated scenes, and providing a promising direction for 3D scene synthesis.

Problem

• Novel approach for indoor scene synthesis using cuboids
• Reduces object intersections in 3D scene generation
• Introduces refined dataset 3DFRONT-NC for better quality

Innovation

• Uses cuboids for compact 3D object representation
• Employs autoregressive model for sequential cuboid arrangement
• Introduces refined 3DFRONT-NC dataset to reduce noise