Learning to Deblur Polarized Images

2024-02-28

arXiv.org

AI Summary

Polarization images are highly susceptible to motion blur induced by camera shake, severely degrading the Degree of Linear Polarization (DoLP) and Angle of Linear Polarization (AoLP). Existing deblurring methods neglect fundamental polarization physical constraints, limiting their performance. To address this, we propose the first polarization-aware single-image deblurring method. Our approach decouples the ill-posed problem into two well-posed subtasks: polarization consistency and image sharpness. We design a two-stage neural network that explicitly enforces DoLP/AoLP consistency constraints and jointly optimizes both the polarization physical model and end-to-end image reconstruction. Extensive experiments on synthetic and real-world polarization datasets demonstrate state-of-the-art performance. Our method significantly improves DoLP and AoLP estimation accuracy and enhances the robustness and accuracy of downstream polarization-based vision tasks—including haze removal and specular reflection suppression.

Technology Category

• Computer Vision: Computational Photography, Image & Video Synthesis
• Machine Learning: Hardware-aware ML
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Social Networks and Social Media: Media and governance, opinion dynamics, filter bubbles, polarization

Abstract

A polarization camera can capture four linear polarized images with different polarizer angles in a single shot, which is useful in polarization-based vision applications since the degree of linear polarization (DoLP) and the angle of linear polarization (AoLP) can be directly computed from the captured polarized images. However, since the on-chip micro-polarizers block part of the light so that the sensor often requires a longer exposure time, the captured polarized images are prone to motion blur caused by camera shakes, leading to noticeable degradation in the computed DoLP and AoLP. Deblurring methods for conventional images often show degraded performance when handling the polarized images since they only focus on deblurring without considering the polarization constraints. In this paper, we propose a polarized image deblurring pipeline to solve the problem in a polarization-aware manner by adopting a divide-and-conquer strategy to explicitly decompose the problem into two less ill-posed sub-problems, and design a two-stage neural network to handle the two sub-problems respectively. Experimental results show that our method achieves state-of-the-art performance on both synthetic and real-world images, and can improve the performance of polarization-based vision applications such as image dehazing and reflection removal.

Problem

• Deblurring polarized images degraded by motion blur
• Addressing polarization constraints in image deblurring
• Improving accuracy of DoLP and AoLP computations

Innovation

• Uses polarization-aware deblurring pipeline
• Divides problem into two sub-problems
• Employs two-stage neural network