RoomPainter: View-Integrated Diffusion for Consistent Indoor Scene Texturing

2024-12-21

arXiv.org

AI Summary

Indoor scene texture synthesis faces challenges including inter-view inconsistency, visible seams, and high computational overhead. This paper introduces RoomPainter, the first zero-shot 2D diffusion-based framework for indoor texture synthesis. It achieves efficient, high-fidelity, and geometrically consistent 3D textures via a global–local two-stage strategy. In the first stage, multi-view integrated sampling (MVIS) combined with neighborhood-enhanced attention generates holistic textures. In the second stage, a repaint-based multi-view refinement strategy (MVRS) performs occlusion-aware, instance-level fine-tuning. Crucially, RoomPainter requires no fine-tuning or iterative optimization. It significantly reduces computational cost while achieving state-of-the-art visual quality, inter-view consistency, and seamless texture synthesis.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Responsible Web: Data and user privacy-enhancing technologies for the Web
• Economics, Online Markets and Human Computation: Uses of LLMs and GenAI for marketplace design, bidding, and strategic interactions
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Indoor scene texture synthesis has garnered significant interest due to its important potential applications in virtual reality, digital media and creative arts. Existing diffusion-model-based researches either rely on per-view inpainting techniques, which are plagued by severe cross-view inconsistencies and conspicuous seams, or adopt optimization-based approaches that involve substantial computational overhead. In this work, we present RoomPainter, a framework that seamlessly integrates efficiency and consistency to achieve high-fidelity texturing of indoor scenes. The core of RoomPainter features a zero-shot technique that effectively adapts a 2D diffusion model for 3D-consistent texture synthesis, along with a two-stage generation strategy that ensures both global and local consistency. Specifically, we introduce Attention-Guided Multi-View Integrated Sampling (MVIS) combined with a neighbor-integrated attention mechanism for zero-shot texture map generation. Using the MVIS, we firstly generate texture map for the entire room to ensure global consistency, then adopt its variant, namely Attention-Guided Multi-View Integrated Repaint Sampling (MVRS) to repaint individual instances within the room, thereby further enhancing local consistency and addressing the occlusion problem. Experiments demonstrate that RoomPainter achieves superior performance for indoor scene texture synthesis in visual quality, global consistency and generation efficiency.

Problem

• Achieving cross-view consistency in indoor scene texturing
• Reducing computational overhead in texture synthesis
• Enhancing global and local texture consistency efficiently

Innovation

• Zero-shot 2D diffusion for 3D textures
• Two-stage global-local consistency strategy
• Attention-guided multi-view integrated sampling