Single-Shot Shape and Reflectance with Spatial Polarization Multiplexing

2025-04-17

AI Summary

This work addresses the challenges of jointly reconstructing 3D shape and spatially varying reflectance from single-shot polarization imaging, particularly under dynamic surface conditions. We propose Spatial Polarization Multiplexing (SPM), which encodes the Angle of Linear Polarization (AoLP) via a constrained de Bruijn sequence across the spatial domain, enabling simultaneous topographic reconstruction and local ellipsometric measurement within a single captured frame. By integrating linear and elliptical polarization models, our method jointly inverts BRDF parameters and the full Mueller matrix. Experiments on real-world data demonstrate sub-millimeter geometric accuracy and comprehensive reflectance recovery. Notably, this is the first approach to achieve single-frame, joint geometry–reflectance reconstruction for dynamic surfaces. The method significantly enhances the robustness and practicality of structured-light imaging on non-ideal (e.g., glossy, anisotropic) and time-varying surfaces.

Technology Category

• Computer Vision: 3D Computer Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Machine Learning: Multimodal Learning

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

We propose spatial polarization multiplexing (SPM) for reconstructing object shape and reflectance from a single polarimetric image and demonstrate its application to dynamic surface recovery. Although single-pattern structured light enables single-shot shape reconstruction, the reflectance is challenging to recover due to the lack of angular sampling of incident light and the entanglement of the projected pattern and the surface color texture. We design a spatially multiplexed pattern of polarization that can be robustly and uniquely decoded for shape reconstruction by quantizing the AoLP values. At the same time, our spatial-multiplexing enables single-shot ellipsometry of linear polarization by projecting differently polarized light within a local region, which separates the specular and diffuse reflections for BRDF estimation. We achieve this spatial polarization multiplexing with a constrained de Bruijn sequence. Unlike single-pattern structured light with intensity and color, our polarization pattern is invisible to the naked eye and retains the natural surface appearance which is essential for accurate appearance modeling and also interaction with people. We experimentally validate our method on real data. The results show that our method can recover the shape, the Mueller matrix, and the BRDF from a single-shot polarimetric image. We also demonstrate the application of our method to dynamic surfaces.

Problem

• Reconstruct object shape and reflectance from single polarimetric image
• Decode spatially multiplexed polarization pattern for shape reconstruction
• Estimate BRDF by separating specular and diffuse reflections

Innovation

• Spatial polarization multiplexing for single-shot reconstruction
• Quantized AoLP values enable robust shape decoding
• Constrained de Bruijn sequence for invisible polarization pattern