Pro-DG: Procedural Diffusion Guidance for Architectural Facade Generation

2025-04-02

AI Summary

This work addresses the challenge of jointly achieving controllability, photorealism, and local fidelity in architectural façade generation. We propose the first framework integrating neural-symbolic procedural shape grammars with diffusion models. Methodologically, we design a multi-level structural alignment mechanism: shape grammars generate semantically controlled layout maps to guide the diffusion process, while jointly optimizing image reconstruction and structure-aware edits (e.g., floor replication, window rearrangement). Our key contribution is the first incorporation of interpretable, procedural grammars into diffusion-based generation—enabling fine-grained semantic control without compromising photographic visual quality. Crucially, identity-preserving edits achieve significantly higher geometric precision than prior approaches. Quantitative evaluation and user studies demonstrate superior performance over baselines—including inpainting—in controllability, fidelity, and practical utility.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Humans and AI: Game Design — Procedural Content Generation & Storytelling
• Machine Learning: Deep Neural Architectures and Foundation Models

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

We present Pro-DG, a framework for procedurally controllable photo-realistic facade generation that combines a procedural shape grammar with diffusion-based image synthesis. Starting from a single input image, we reconstruct its facade layout using grammar rules, then edit that structure through user-defined transformations. As facades are inherently multi-hierarchical structures, we introduce hierarchical matching procedure that aligns facade structures at different levels which is used to introduce control maps to guide a generative diffusion pipeline. This approach retains local appearance fidelity while accommodating large-scale edits such as floor duplication or window rearrangement. We provide a thorough evaluation, comparing Pro-DG against inpainting-based baselines and synthetic ground truths. Our user study and quantitative measurements indicate improved preservation of architectural identity and higher edit accuracy. Our novel method is the first to integrate neuro-symbolically derived shape-grammars for modeling with modern generative model and highlights the broader potential of such approaches for precise and controllable image manipulation.

Problem

• Procedural control for photo-realistic facade generation
• Hierarchical alignment of multi-level facade structures
• Integration of shape-grammars with generative diffusion models

Innovation

• Combines shape grammar with diffusion synthesis
• Hierarchical matching for multi-level facade control
• Neuro-symbolic integration for precise image manipulation