Frequency-Domain Fusion Transformer for Image Inpainting

2025-06-23

AI Summary

Image inpainting remains challenging for reconstructing complex textures and restoring large occluded regions, suffering from high-frequency detail loss and excessive computational cost. To address these issues, we propose a frequency-domain enhanced Transformer framework. First, we design a wavelet-Gabor fused attention mechanism to explicitly model multi-scale structural patterns. Second, we introduce learnable FFT-based frequency-domain filters to adaptively preserve high-frequency components while suppressing noise. Third, we construct a four-stage encoder-decoder architecture, jointly optimized with a composite loss function that balances global semantic coherence and local detail fidelity. Extensive experiments demonstrate that our method achieves superior performance over state-of-the-art approaches in both quantitative metrics (PSNR/SSIM) and visual quality, with enhanced high-frequency detail preservation and approximately 23% faster inference speed—effectively reconciling restoration accuracy and computational efficiency.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.

Application Category

• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

Image inpainting plays a vital role in restoring missing image regions and supporting high-level vision tasks, but traditional methods struggle with complex textures and large occlusions. Although Transformer-based approaches have demonstrated strong global modeling capabilities, they often fail to preserve high-frequency details due to the low-pass nature of self-attention and suffer from high computational costs. To address these challenges, this paper proposes a Transformer-based image inpainting method incorporating frequency-domain fusion. Specifically, an attention mechanism combining wavelet transform and Gabor filtering is introduced to enhance multi-scale structural modeling and detail preservation. Additionally, a learnable frequency-domain filter based on the fast Fourier transform is designed to replace the feedforward network, enabling adaptive noise suppression and detail retention. The model adopts a four-level encoder-decoder structure and is guided by a novel loss strategy to balance global semantics and fine details. Experimental results demonstrate that the proposed method effectively improves the quality of image inpainting by preserving more high-frequency information.

Problem

• Restoring missing image regions with complex textures
• Preserving high-frequency details in Transformer-based inpainting
• Reducing computational costs in image inpainting methods

Innovation

• Wavelet and Gabor attention for multi-scale detail
• Learnable FFT filter for adaptive noise control
• Four-level encoder-decoder with balanced loss strategy