PIDSR:ComplementaryPolarizedImageDemosaicingandSuper-Resolution

2025-04-10

AI Summary

Polarization cameras capture low-resolution color-polarization filter array (CPFA) raw images; conventional sequential demosaicking (PID) followed by polarization image super-resolution (PISR) amplifies errors in degree of polarization (DoP) and angle of polarization (AoP). To address this, we propose the first jointly optimized demosaicking and super-resolution framework that avoids error accumulation through complementary modeling. Our method introduces a multi-branch feature collaboration network integrating (i) joint color-polarization representation, (ii) cross-angle polarization consistency constraints, and (iii) frequency-domain enhancement modules, enabling end-to-end reconstruction of high-resolution (HR) polarization images. Extensive experiments on both synthetic and real-world datasets demonstrate state-of-the-art performance: DoP and AoP estimation errors are reduced by over 40% compared to prior methods. Moreover, the improved polarization fidelity significantly enhances downstream task robustness, including material classification and 3D reconstruction.

Technology Category

• Computer Vision: Learning & Optimization for CV
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Machine Learning: Multimodal Learning

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Polarization cameras can capture multiple polarized images with different polarizer angles in a single shot, bringing convenience to polarization-based downstream tasks. However, their direct outputs are color-polarization filter array (CPFA) raw images, requiring demosaicing to reconstruct full-resolution, full-color polarized images; unfortunately, this necessary step introduces artifacts that make polarization-related parameters such as the degree of polarization (DoP) and angle of polarization (AoP) prone to error. Besides, limited by the hardware design, the resolution of a polarization camera is often much lower than that of a conventional RGB camera. Existing polarized image demosaicing (PID) methods are limited in that they cannot enhance resolution, while polarized image super-resolution (PISR) methods, though designed to obtain high-resolution (HR) polarized images from the demosaicing results, tend to retain or even amplify errors in the DoP and AoP introduced by demosaicing artifacts. In this paper, we propose PIDSR, a joint framework that performs complementary Polarized Image Demosaicing and Super-Resolution, showing the ability to robustly obtain high-quality HR polarized images with more accurate DoP and AoP from a CPFA raw image in a direct manner. Experiments show our PIDSR not only achieves state-of-the-art performance on both synthetic and real data, but also facilitates downstream tasks.

Problem

• Reconstructs full-resolution polarized images from CPFA raw data
• Reduces artifacts in polarization parameters (DoP and AoP)
• Enhances resolution while maintaining polarization accuracy

Innovation

• Joint demosaicing and super-resolution for polarized images
• Direct high-quality reconstruction from CPFA raw data
• Improved accuracy in polarization parameters (DoP/AoP)