Sketch-Based Facade Renovation With Generative AI: A Streamlined Framework for Bypassing As-Built Modelling in Industrial Adaptive Reuse

2026-01-13

AI Summary

This study addresses the limitations of conventional façade renovation approaches, which rely on labor-intensive as-is modeling and struggle to balance structural preservation with design innovation. To overcome this, the authors propose a novel three-stage generative framework that integrates a fine-tuned vision-language model (VLM) with Stable Diffusion, ControlNet, and image inpainting techniques. This framework enables end-to-end generation of high-fidelity façade renovation proposals directly from rough sketches and textual descriptions, eliminating the need for detailed geometric modeling. Experimental results on a real-world industrial building dataset demonstrate that the method effectively preserves original structural characteristics while significantly enhancing façade detail quality, thereby supporting efficient design iteration and accurate expression of architectural intent.

Technology Category

• Computer Vision: Generative Adversarial Networks (GANs) for Vision
• Natural Language Processing: Generation
• Machine Learning: Deep Neural Architectures and Foundation Models

Application Category

• Social Networks and Social Media: Generative AI / large language models and their impact on social systems
• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Facade renovation offers a more sustainable alternative to full demolition, yet producing design proposals that preserve existing structures while expressing new intent remains challenging. Current workflows typically require detailed as-built modelling before design, which is time-consuming, labour-intensive, and often involves repeated revisions. To solve this issue, we propose a three-stage framework combining generative artificial intelligence (AI) and vision-language models (VLM) that directly processes rough structural sketch and textual descriptions to produce consistent renovation proposals. First, the input sketch is used by a fine-tuned VLM model to predict bounding boxes specifying where modifications are needed and which components should be added. Next, a stable diffusion model generates detailed sketches of new elements, which are merged with the original outline through a generative inpainting pipeline. Finally, ControlNet is employed to refine the result into a photorealistic image. Experiments on datasets and real industrial buildings indicate that the proposed framework can generate renovation proposals that preserve the original structure while improving facade detail quality. This approach effectively bypasses the need for detailed as-built modelling, enabling architects to rapidly explore design alternatives, iterate on early-stage concepts, and communicate renovation intentions with greater clarity.

Problem

• Facade renovation
• As-built modelling
• Generative AI
• Adaptive reuse
• Design proposal

Innovation

• generative AI
• vision-language model
• facade renovation
• as-built modelling bypass
• ControlNet