StructDiff: A Structure-Preserving and Spatially Controllable Diffusion Model for Single-Image Generation

2026-04-14

AI Summary

This work addresses the challenge of preserving structural layout in single-image generation, particularly in scenes containing large rigid objects or strong spatial constraints, where existing methods often lack control over the position and scale of generated content. To overcome this, the authors propose StructDiff, which introduces 3D positional encoding as a spatial prior—the first such use in single-image generation—and integrates an adaptive receptive field module with a single-scale diffusion model to enable flexible control over object location, scale, and fine details. Furthermore, they present a novel evaluation metric grounded in large language models to alleviate limitations of current quantitative measures and reduce reliance on costly human assessments. The method demonstrates significant improvements over state-of-the-art approaches in structural consistency, visual fidelity, and spatial controllability, with successful applications in text-guided generation, image editing, outpainting, and sketch-to-image synthesis.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Natural Language Processing: Generation
• Machine Learning: Deep Generative Models & Autoencoders

Application Category

• Search and Retrieval-Augmented AI: Large language models for search
• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

This paper introduces StructDiff, a generative framework based on a single-scale diffusion model for single-image generation. Single-image generation aims to synthesize diverse samples with similar visual content to the source image by capturing its internal statistics, without relying on external data. However, existing methods often struggle to preserve the structural layout, especially for images with large rigid objects or strict spatial constraints. Moreover, most approaches lack spatial controllability, making it difficult to guide the structure or placement of generated content. To address these challenges, StructDiff introduces an \textit{adaptive receptive field} module to maintain both global and local distributions. Building on this foundation, StructDiff incorporates 3D positional encoding (PE) as a spatial prior, allowing flexible control over positions, scale, and local details of generated objects. To our knowledge, this spatial control capability represents the first exploration of PE-based manipulation in single-image generation. Furthermore, we propose a novel evaluation criterion for single-image generation based on large language models (LLMs). This criterion specifically addresses the limitations of existing objective metrics and the high labor costs associated with user studies. StructDiff also demonstrates broad applicability across downstream tasks, such as text-guided image generation, image editing, outpainting, and paint-to-image synthesis. Extensive experiments demonstrate that StructDiff outperforms existing methods in structural consistency, visual quality, and spatial controllability. The project page is available at https://butter-crab.github.io/StructDiff/.

Problem

• single-image generation
• structural preservation
• spatial controllability
• diffusion model

Innovation

• adaptive receptive field
• 3D positional encoding
• spatial controllability
• single-image generation
• structure-preserving diffusion