Implicit Image-to-Image Schrodinger Bridge for Image Restoration

2024-03-10

AI Summary

Diffusion-based image restoration suffers from slow inference; although I²SB improves efficiency, further acceleration is needed. This paper proposes Implicit Image-to-Image Schrödinger Bridge (I³SB), the first non-Markovian generative framework for restoration: at each step, the initial degraded image is explicitly injected as an implicit condition, enabling zero-shot adaptation of pre-trained I²SB models without retraining while preserving marginal distribution consistency. I³SB integrates Schrödinger bridge theory, score-based modeling, and implicit conditional injection to jointly leverage score-based priors and degradation guidance. Evaluated on diverse degradation tasks—including medical, facial, and natural images—I³SB achieves perceptual quality comparable to I²SB using significantly fewer sampling steps, while substantially improving reconstruction realism and structural fidelity.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Deep Generative Models & Autoencoders
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Diffusion-based models have demonstrated remarkable effectiveness in image restoration tasks; however, their iterative denoising process, which starts from Gaussian noise, often leads to slow inference speeds. The Image-to-Image Schr""odinger Bridge (I$^2$SB) offers a promising alternative by initializing the generative process from corrupted images while leveraging training techniques from score-based diffusion models. In this paper, we introduce the Implicit Image-to-Image Schr""odinger Bridge (I$^3$SB) to further accelerate the generative process of I$^2$SB. I$^3$SB restructures the generative process into a non-Markovian framework by incorporating the initial corrupted image at each generative step, effectively preserving and utilizing its information. To enable direct use of pretrained I$^2$SB models without additional training, we ensure consistency in marginal distributions. Extensive experiments across many image corruptions, including noise, low resolution, JPEG compression, and sparse sampling, and multiple image modalities, such as natural, human face, and medical images, demonstrate the acceleration benefits of I$^3$SB. Compared to I$^2$SB, I$^3$SB achieves the same perceptual quality with fewer generative steps, while maintaining or improving fidelity to the ground truth.

Problem

• Accelerate image restoration using implicit Schrodinger Bridge
• Reduce generative steps while maintaining perceptual quality
• Apply non-Markovian framework to preserve corrupted image information

Innovation

• Implicit Image-to-Image Schrodinger Bridge
• Non-Markovian generative framework
• Consistent marginal distributions