Inferring geometry and material properties from Mueller matrices with machine learning

2025-08-27

AI Summary

This study addresses the ill-posedness inherent in jointly inverting Mueller matrices to recover both surface geometry and material properties. We propose an end-to-end deep learning framework leveraging full-angle Mueller matrices acquired across multiple wavelengths (450–650 nm). The model incorporates physics-based optical priors to simultaneously estimate high-fidelity surface normal fields, reconstruct 3D geometry, and classify material types. Key insights reveal that diagonal Mueller matrix elements primarily govern material discrimination, whereas off-diagonal elements encode geometric information—enabling synergistic, rather than isolated, exploitation of both. Experiments demonstrate sub-degree normal estimation and millimeter-level geometric reconstruction even on objects with unseen materials, alongside >95% material classification accuracy. This work provides the first systematic validation of the Mueller matrix’s information completeness and feasibility for joint cross-modal perception.

Technology Category

• Machine Learning: Matrix & Tensor Methods
• Computer Vision: Multi-modal Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Mueller matrices (MMs) encode information on geometry and material properties, but recovering both simultaneously is an ill-posed problem. We explore whether MMs contain sufficient information to infer surface geometry and material properties with machine learning. We use a dataset of spheres of various isotropic materials, with MMs captured over the full angular domain at five visible wavelengths (450-650 nm). We train machine learning models to predict material properties and surface normals using only these MMs as input. We demonstrate that, even when the material type is unknown, surface normals can be predicted and object geometry reconstructed. Moreover, MMs allow models to identify material types correctly. Further analyses show that diagonal elements are key for material characterization, and off-diagonal elements are decisive for normal estimation.

Problem

• Recovering surface geometry and material properties simultaneously from Mueller matrices
• Determining if machine learning can infer both properties using Mueller matrices
• Predicting material types and reconstructing object geometry from Mueller matrices

Innovation

• Machine learning models predict material properties from Mueller matrices
• Surface normals reconstructed using Mueller matrices as input
• Diagonal and off-diagonal elements enable material and geometry inference