Light of Normals: Unified Feature Representation for Universal Photometric Stereo

2025-06-23

AI Summary

General-purpose photometric stereo (PS) faces two key challenges: (1) strong entanglement between illumination variations and surface normal features, hindering disambiguation of brightness changes; and (2) loss of high-frequency geometric details—such as self-shadowing, inter-reflections, and subtle normal variations—during feature processing. To address these, we propose a physics-inspired unified feature representation framework that explicitly decouples joint illumination-normal modeling to suppress coupling interference. Furthermore, we introduce a multi-scale contextual aggregation module and a residual detail enhancement module to strengthen high-frequency geometric structure encoding. Our method achieves state-of-the-art performance across multiple general-purpose PS benchmarks, significantly improving normal estimation accuracy and fine-detail reconstruction quality—particularly in regions with severe self-shadowing and complex material properties.

Technology Category

• Computer Vision: Low Level & Physics-based Vision
• Machine Learning: Feature Construction/Reformulation
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Security and Privacy: Large-scale security measurements

Abstract

Universal photometric stereo (PS) aims to recover high-quality surface normals from objects under arbitrary lighting conditions without relying on specific illumination models. Despite recent advances such as SDM-UniPS and Uni MS-PS, two fundamental challenges persist: 1) the deep coupling between varying illumination and surface normal features, where ambiguity in observed intensity makes it difficult to determine whether brightness variations stem from lighting changes or surface orientation; and 2) the preservation of high-frequency geometric details in complex surfaces, where intricate geometries create self-shadowing, inter-reflections, and subtle normal variations that conventional feature processing operations struggle to capture accurately.

Problem

• Decoupling illumination and surface normal features ambiguity
• Preserving high-frequency geometric details in complex surfaces
• Handling self-shadowing and inter-reflections in photometric stereo

Innovation

• Unified feature representation for arbitrary lighting
• Decoupling illumination and surface normal features
• Preserving high-frequency geometric details accurately