A Principled Self-Referenced Early Stopping Approach for Deep Image Prior

2026-05-24

AI Summary

This work addresses the instability of Deep Image Prior (DIP) in image reconstruction under noise overfitting when no training data are available. The authors propose a novel early-stopping framework based on self-referenced pseudo-images, which introduces the concept of dual-noise observation into DIP for the first time. By constructing a self-referenced pseudo-image from a single observation, the method enables overfitting detection without requiring accurate noise estimation. Leveraging analysis of running variance fluctuations, three early-stopping strategies tailored to inverse imaging problems are developed. Extensive experiments demonstrate that the proposed approach consistently outperforms existing DIP-based early-stopping methods across various noise types and levels, achieving superior performance in both natural image restoration and medical image reconstruction tasks.

Abstract

Recently, Deep Image Prior (DIP) has demonstrated strong capabilities for solving inverse imaging problems (IIPs) by optimizing a randomly initialized convolutional neural network in a training-data-free regime. However, DIP suffers from overfitting to noisy measurements due to network over-parameterization, making early stopping (ES) essential. The most successful ES method tracks fluctuations in the running variance of the network output to detect overfitting. However, in many applications, these fluctuations may appear prematurely, leading to unstable reconstructions. In this paper, we first show that nearly optimal DIP early stopping can be achieved when two independent noisy copies of the degraded image are available. Motivated by this observation, and since obtaining two fully independent copies is infeasible, we propose an overfitting detection framework based on constructing pseudo self-referenced images, resulting in three IIP-specific algorithms. Our approach is further supported by theoretical results on single-reference validation, pseudo-validation estimation, and the impact of shared noise. Across different IIPs, ranging from natural image restoration to medical image reconstruction, and under varying noise levels and noise types, our methods consistently outperform existing DIP early stopping approaches, all without requiring an accurate estimate of the noise level.

Problem

• Deep Image Prior
• inverse imaging problems
• early stopping
• overfitting
• noise

Innovation

• Deep Image Prior
• Early Stopping
• Self-Referenced Validation
• Overfitting Detection
• Inverse Imaging Problems