Denoising as Adaptation: Noise-Space Domain Adaptation for Image Restoration

2024-06-26

arXiv.org

AI Summary

This paper addresses the poor generalization of learning-based image restoration methods in real-world scenarios—a limitation stemming from significant domain shift between synthetic training data and real images. To bridge this gap, we propose a novel domain adaptation paradigm tailored to the noise space of diffusion models. Our key contributions are: (1) the first “denoising-as-adaptation” mechanism, which progressively aligns restoration outputs of synthetic and real images toward the clean distribution via multi-step conditional denoising and a domain-aligned diffusion loss; and (2) a channel-shuffling layer coupled with residual-swap contrastive learning to implicitly blur domain boundaries and suppress shortcut feature dependencies. Evaluated on denoising, deblurring, and deraining tasks, our method substantially outperforms existing domain-adaptive and blind restoration approaches, achieving state-of-the-art generalization performance on real-world images.

Technology Category

• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Computer Vision: Diffusion Models for Vision
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Vertical and domain-specific search
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization

Abstract

Although learning-based image restoration methods have made significant progress, they still struggle with limited generalization to real-world scenarios due to the substantial domain gap caused by training on synthetic data. Existing methods address this issue by improving data synthesis pipelines, estimating degradation kernels, employing deep internal learning, and performing domain adaptation and regularization. Previous domain adaptation methods have sought to bridge the domain gap by learning domain-invariant knowledge in either feature or pixel space. However, these techniques often struggle to extend to low-level vision tasks within a stable and compact framework. In this paper, we show that it is possible to perform domain adaptation via the noise space using diffusion models. In particular, by leveraging the unique property of how auxiliary conditional inputs influence the multi-step denoising process, we derive a meaningful diffusion loss that guides the restoration model in progressively aligning both restored synthetic and real-world outputs with a target clean distribution. We refer to this method as denoising as adaptation. To prevent shortcuts during joint training, we present crucial strategies such as channel-shuffling layer and residual-swapping contrastive learning in the diffusion model. They implicitly blur the boundaries between conditioned synthetic and real data and prevent the reliance of the model on easily distinguishable features. Experimental results on three classical image restoration tasks, namely denoising, deblurring, and deraining, demonstrate the effectiveness of the proposed method.

Problem

• Addressing domain gap in image restoration
• Enhancing generalization with noise-space adaptation
• Improving real-world image restoration tasks

Innovation

• Domain adaptation via noise space
• Diffusion models for image restoration
• Channel-shuffling layer prevents shortcuts