CARD: Correlation Aware Restoration with Diffusion

2025-12-04

AI Summary

Real-world image sensors often introduce spatially correlated noise, whereas mainstream restoration methods assume independent and identically distributed (i.i.d.) Gaussian noise—leading to suboptimal performance in practice. To address this, we propose the first diffusion-based framework that explicitly models correlated Gaussian noise, featuring a training-free noise whitening scheme. Our approach employs an invertible whitening preprocessing step to decouple noise correlations and integrates a whitening-aware update mechanism into the diffusion sampling process, enabling efficient inference via closed-form solutions. We further introduce CIN-D, the first benchmark specifically designed for evaluating restoration under rolling-shutter sensor-induced correlated noise. Extensive experiments demonstrate that our method achieves state-of-the-art performance across denoising, deblurring, and super-resolution tasks on both synthetic and real-world data, validating its effectiveness and strong generalization capability.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Adversarial Learning & Robustness
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Denoising diffusion models have achieved state-of-the-art performance in image restoration by modeling the process as sequential denoising steps. However, most approaches assume independent and identically distributed (i.i.d.) Gaussian noise, while real-world sensors often exhibit spatially correlated noise due to readout mechanisms, limiting their practical effectiveness. We introduce Correlation Aware Restoration with Diffusion (CARD), a training-free extension of DDRM that explicitly handles correlated Gaussian noise. CARD first whitens the noisy observation, which converts the noise into an i.i.d. form. Then, the diffusion restoration steps are replaced with noise-whitened updates, which inherits DDRM's closed-form sampling efficiency while now being able to handle correlated noise. To emphasize the importance of addressing correlated noise, we contribute CIN-D, a novel correlated noise dataset captured across diverse illumination conditions to evaluate restoration methods on real rolling-shutter sensor noise. This dataset fills a critical gap in the literature for experimental evaluation with real-world correlated noise. Experiments on standard benchmarks with synthetic correlated noise and on CIN-D demonstrate that CARD consistently outperforms existing methods across denoising, deblurring, and super-resolution tasks.

Problem

• Handles spatially correlated noise in image restoration
• Introduces training-free diffusion method for correlated Gaussian noise
• Provides real-world dataset for evaluating correlated noise restoration

Innovation

• Whitening noisy observations for i.i.d. conversion
• Using noise-whitened updates in diffusion restoration
• Introducing CIN-D dataset for real correlated noise evaluation