PolarAPP: Beyond Polarization Demosaicking for Polarimetric Applications

2026-03-24

AI Summary

Existing polarization imaging methods optimize only photometric fidelity during demosaicking, disregarding the requirements of downstream tasks, which leads to incomplete reconstructions and suboptimal performance. This work proposes PolarAPP, a novel framework that, for the first time, enables end-to-end joint optimization of polarization demosaicking and downstream tasks. PolarAPP ensures semantic consistency through a task-oriented feature alignment mechanism and introduces an equivalent imaging constraint that obviates data rearrangement while directly regressing physically meaningful outputs. Task-specific fine-tuning further enhances accuracy. Experiments demonstrate that the proposed method significantly outperforms existing approaches in both demosaicking quality and multiple downstream tasks, thereby transcending the conventional task-agnostic reconstruction paradigm.

Technology Category

• Computer Vision: Learning & Optimization for CV
• Machine Learning: Feature Construction/Reformulation
• Constraint Satisfaction and Optimization: Applications

Application Category

• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Security and Privacy: Security and privacy of machine learning and AI applications
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Polarimetric imaging enables advanced vision applications such as normal estimation and de-reflection by capturing unique surface-material interactions. However, existing applications (alternatively called downstream tasks) rely on datasets constructed by naively regrouping raw measurements from division-of-focal-plane sensors, where pixels of the same polarization angle are extracted and aligned into sparse images without proper demosaicking. This reconstruction strategy results in suboptimal, incomplete targets that limit downstream performance. Moreover, current demosaicking methods are task-agnostic, optimizing only for photometric fidelity rather than utility in downstream tasks. Towards this end, we propose PolarAPP, the first framework to jointly optimize demosaicking and its downstream tasks. PolarAPP introduces a feature alignment mechanism that semantically aligns the representations of demosaicking and downstream networks via meta-learning, guiding the reconstruction to be task-aware. It further employs an equivalent imaging constraint for demosaicking training, enabling direct regression to physically meaningful outputs without relying on rearranged data. Finally, a task-refinement stage fine-tunes the task network using the stable demosaicking front-end to further enhance accuracy. Extensive experimental results demonstrate that PolarAPP outperforms existing methods in both demosaicking quality and downstream performance. Code is available upon acceptance.

Problem

• polarimetric imaging
• demosaicking
• downstream tasks
• division-of-focal-plane
• task-aware reconstruction

Innovation

• task-aware demosaicking
• polarimetric imaging
• feature alignment
• meta-learning
• equivalent imaging constraint