RGB-to-Polarization Estimation: A New Task and Benchmark Study

2025-05-19

AI Summary

Polarimetric imaging traditionally relies on specialized hardware, incurring high costs and deployment challenges. To address this, this paper introduces and systematically investigates the novel task of “RGB-to-polarization image estimation”—recovering physically consistent Stokes parameters (S₀, S₁, S₂) and degree/polarization angle (DoP/AoP) solely from standard RGB inputs. We establish the first open-source, comprehensive benchmark, comprising multi-source real-world polarization data, rigorously partitioned training/validation/test splits, and a unified evaluation protocol. Our benchmark comprehensively assesses reconstruction- and generation-based models, as well as dedicated supervised learning versus large-model transfer paradigms. Experiments span 12 state-of-the-art image restoration and generation architectures, evaluated via quantitative metrics (PSNR, SSIM, LPIPS) and physical consistency analysis. Results establish current performance limits and reveal fundamental trade-offs among accuracy, generalization, and physical plausibility—providing a reproducible foundation and technical roadmap for lightweight, RGB-driven polarimetric perception.

Technology Category

• Computer Vision: Computational Photography, Image & Video Synthesis
• Intelligent Robots: State Estimation
• Machine Learning: Adversarial Learning & Robustness

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

Polarization images provide rich physical information that is fundamentally absent from standard RGB images, benefiting a wide range of computer vision applications such as reflection separation and material classification. However, the acquisition of polarization images typically requires additional optical components, which increases both the cost and the complexity of the applications. To bridge this gap, we introduce a new task: RGB-to-polarization image estimation, which aims to infer polarization information directly from RGB images. In this work, we establish the first comprehensive benchmark for this task by leveraging existing polarization datasets and evaluating a diverse set of state-of-the-art deep learning models, including both restoration-oriented and generative architectures. Through extensive quantitative and qualitative analysis, our benchmark not only establishes the current performance ceiling of RGB-to-polarization estimation, but also systematically reveals the respective strengths and limitations of different model families -- such as direct reconstruction versus generative synthesis, and task-specific training versus large-scale pre-training. In addition, we provide some potential directions for future research on polarization estimation. This benchmark is intended to serve as a foundational resource to facilitate the design and evaluation of future methods for polarization estimation from standard RGB inputs.

Problem

• Estimating polarization information from RGB images
• Benchmarking deep learning models for RGB-to-polarization task
• Analyzing strengths and limitations of different model families

Innovation

• RGB-to-polarization image estimation task
• Comprehensive benchmark with diverse models
• Direct reconstruction versus generative synthesis