Reblurring-Guided Single Image Defocus Deblurring: A Learning Framework with Misaligned Training Pairs

2024-09-26

arXiv.org

AI Summary

In single-image defocus deblurring, acquiring strictly aligned blurry/sharp image pairs—or triplets—is challenging in real-world scenarios, and spatial misalignment further complicates supervision. Method: This paper proposes a refocusing-guided learning framework that trains solely on misaligned image pairs. It introduces a novel refocusing consistency constraint to implicitly convert misaligned pairs into pseudo-triplets. A differentiable refocusing module generates spatially varying pseudo-defocus maps; combined with self-supervised blur kernel estimation and pseudo-label distillation, it eliminates the need for ground-truth kernel annotations. The backbone network is designed based on spatially varying degradation priors. Contribution/Results: Evaluated on a newly constructed real-world SDD dataset featuring typical misalignments, our method significantly outperforms existing approaches, achieving high-fidelity restoration. It establishes a new paradigm for misalignment-aware supervised learning and provides the first dedicated benchmark for this problem.

Technology Category

• Computer Vision: Learning & Optimization for CV
• Search and Optimization: Learning to Search
• Machine Learning: Multi-instance/Multi-view Learning

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

For single image defocus deblurring, acquiring well-aligned training pairs (or training triplets), i.e., a defocus blurry image, an all-in-focus sharp image (and a defocus blur map), is an intricate task for the development of deblurring models. Existing image defocus deblurring methods typically rely on training data collected by specialized imaging equipment, presupposing that these pairs or triplets are perfectly aligned. However, in practical scenarios involving the collection of real-world data, direct acquisition of training triplets is infeasible, and training pairs inevitably encounter spatial misalignment issues. In this work, we introduce a reblurring-guided learning framework for single image defocus deblurring, enabling the learning of a deblurring network even with misaligned training pairs. Specifically, we first propose a baseline defocus deblurring network that utilizes spatially varying defocus blur map as degradation prior to enhance the deblurring performance. Then, to effectively learn the baseline defocus deblurring network with misaligned training pairs, our reblurring module ensures spatial consistency between the deblurred image, the reblurred image and the input blurry image by reconstructing spatially variant isotropic blur kernels. Moreover, the spatially variant blur derived from the reblurring module can serve as pseudo supervision for defocus blur map during training, interestingly transforming training pairs into training triplets. Additionally, we have collected a new dataset specifically for single image defocus deblurring (SDD) with typical misalignments, which not only substantiates our proposed method but also serves as a benchmark for future research.

Problem

• Addresses misaligned training pairs in defocus deblurring
• Proposes reblurring-guided framework for spatial consistency
• Introduces pseudo supervision for defocus blur estimation

Innovation

• Reblurring-guided learning with misaligned pairs
• Spatially variant isotropic blur kernel reconstruction
• Lightweight defocus blur estimator with fusion