Pooling-Based Context Modeling for Convolution-Free Deep Image Prior

2026-07-03

AI Summary

This work addresses the limitations of traditional Deep Image Prior (DIP)—namely its susceptibility to overfitting noise due to over-parameterization and high computational cost stemming from convolutional operations—by proposing Pool-DIP, a novel convolution-free DIP architecture. Pool-DIP uniquely integrates pooling mechanisms into contextual modeling, leveraging a pure multilayer perceptron combined with pooling operations to efficiently capture spatial information. This design substantially reduces both parameter count and computational complexity while enhancing denoising stability and generalization capability. Spectral analysis further demonstrates its effective regulation of high-frequency components. Extensive experiments show that Pool-DIP achieves competitive denoising performance on multiple synthetic and real-world datasets and successfully generalizes to super-resolution and image inpainting tasks.

Abstract

Convolutional Neural Networks (CNNs) achieve strong denoising performance by exploiting spatial context from neighboring pixels. Deep Image Prior (DIP) leverages this property to restore images from a single noisy input without requiring large datasets. However, the over-parameterized architecture of DIP often leads to noise fitting during optimization. In this paper, we propose Pool-DIP, a convolution-free architecture that incorporates pooling-based contrast modeling to capture spatial context efficiently. Pool-DIP improves denoising performance while significantly reducing the number of parameters and computational complexity compared to convolution-based DIP models. Experimental results show that Pool-DIP achieves competitive performance across multiple datasets, including a real-world benchmark. Spectral analysis further reveals that Pool-DIP stabilizes the evolution of high-frequency components during optimization and suppresses erroneous high-frequency signals. The proposed architecture also generalizes well to other image restoration tasks such as super-resolution and inpainting.

Problem

• Deep Image Prior
• noise fitting
• image denoising
• over-parameterization
• spatial context

Innovation

• convolution-free
• pooling-based context modeling
• Deep Image Prior
• image denoising
• parameter efficiency