PolarFree: Polarization-based Reflection-free Imaging

2025-03-23

AI Summary

To address detail loss and semantic ambiguity caused by strong reflections in real-world scenes, this paper proposes a novel polarization-prior-driven reflection removal paradigm. Methodologically: (1) we introduce PolaRGB—the first large-scale real-world polarization-RGB paired dataset comprising 6,500 image pairs, eight times larger than existing benchmarks; (2) we pioneer the integration of conditional diffusion models into polarization-guided reflection removal, leveraging polarization optical modeling and multimodal alignment to generate high-fidelity reflection-free priors; (3) we design an end-to-end reflection decomposition network for precise reflection-transmission separation. Extensive experiments across diverse indoor and outdoor scenarios demonstrate substantial improvements in image clarity and structural fidelity, establishing new state-of-the-art performance in polarization-based reflection removal. Both code and the PolaRGB dataset are publicly released.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Adversarial Learning & Robustness
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Security and Privacy: Data transparency and provenance

Abstract

Reflection removal is challenging due to complex light interactions, where reflections obscure important details and hinder scene understanding. Polarization naturally provides a powerful cue to distinguish between reflected and transmitted light, enabling more accurate reflection removal. However, existing methods often rely on small-scale or synthetic datasets, which fail to capture the diversity and complexity of real-world scenarios. To this end, we construct a large-scale dataset, PolaRGB, for Polarization-based reflection removal of RGB images, which enables us to train models that generalize effectively across a wide range of real-world scenarios. The PolaRGB dataset contains 6,500 well-aligned mixed-transmission image pairs, 8x larger than existing polarization datasets, and is the first to include both RGB and polarization images captured across diverse indoor and outdoor environments with varying lighting conditions. Besides, to fully exploit the potential of polarization cues for reflection removal, we introduce PolarFree, which leverages diffusion process to generate reflection-free cues for accurate reflection removal. Extensive experiments show that PolarFree significantly enhances image clarity in challenging reflective scenarios, setting a new benchmark for polarized imaging and reflection removal. Code and dataset are available at https://github.com/mdyao/PolarFree.

Problem

• Challenges in reflection removal due to complex light interactions
• Lack of diverse real-world datasets for polarization-based methods
• Need for accurate reflection removal using polarization cues

Innovation

• Large-scale PolaRGB dataset for reflection removal
• PolarFree uses diffusion for reflection-free cues
• Combines RGB and polarization for accurate removal