Thinking inside the Convolution for Image Inpainting: Reconstructing Texture via Structure under Global and Local Side

2026-02-03

AI Summary

Existing image inpainting methods often suffer from the loss of structural and textural details during convolutional downsampling, which limits reconstruction quality. This work presents the first systematic investigation into the complementary relationship between structural and textural features in the downsampling stage and introduces a mutual guidance mechanism based on statistical normalization and denormalization. Integrated within an encoder–decoder framework, this approach enables effective multi-scale feature reconstruction by preserving and recovering critical details. The proposed method consistently outperforms current state-of-the-art techniques on both 256×256 and 512×512 resolution images. Furthermore, replacing the standard encoder with the proposed module yields significant performance gains, demonstrating its effectiveness and generalizability.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Deep Generative Models & Autoencoders
• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.

Application Category

• Web Mining and Content Analysis: Normalization, clustering, classification, and summarization of Web text
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG

Abstract

Image inpainting has earned substantial progress, owing to the encoder-and-decoder pipeline, which is benefited from the Convolutional Neural Networks (CNNs) with convolutional downsampling to inpaint the masked regions semantically from the known regions within the encoder, coupled with an upsampling process from the decoder for final inpainting output. Recent studies intuitively identify the high-frequency structure and low-frequency texture to be extracted by CNNs from the encoder, and subsequently for a desirable upsampling recovery. However, the existing arts inevitably overlook the information loss for both structure and texture feature maps during the convolutional downsampling process, hence suffer from a non-ideal upsampling output. In this paper, we systematically answer whether and how the structure and texture feature map can mutually help to alleviate the information loss during the convolutional downsampling. Given the structure and texture feature maps, we adopt the statistical normalization and denormalization strategy for the reconstruction guidance during the convolutional downsampling process. The extensive experimental results validate its advantages to the state-of-the-arts over the images from low-to-high resolutions including 256*256 and 512*512, especially holds by substituting all the encoders by ours. Our code is available at https://github.com/htyjers/ConvInpaint-TSGL

Problem

• image inpainting
• convolutional downsampling
• structure and texture
• information loss
• feature reconstruction

Innovation

• image inpainting
• convolutional downsampling
• structure-texture interaction
• statistical normalization
• feature reconstruction