Uncertainty for SVBRDF Acquisition using Frequency Analysis

2024-06-25

AI Summary

This work addresses the challenge of quantifying appearance uncertainty in SVBRDF reconstruction from uncontrolled, multi-view lighting captures. We propose the first efficient uncertainty modeling method based on frequency-domain entropy analysis. By introducing frequency-domain analysis—previously unexplored in SVBRDF uncertainty modeling—and accelerating entropy computation, our approach generates full-object uncertainty maps in milliseconds, with results consistent with physically-based path tracing. Theoretical analysis and experiments demonstrate a strong positive correlation between frequency-domain entropy and reconstruction error, enabling effective guidance for acquisition optimization, cross-view surface information sharing, and diffusion-based inpainting. On standard benchmarks, our method achieves state-of-the-art SVBRDF reconstruction accuracy, and the predicted uncertainty maps closely align with ground-truth errors. The implementation is open-sourced and empirically validated.

Technology Category

• Reasoning under Uncertainty: Uncertainty Representations
• Machine Learning: Calibration & Uncertainty Quantification
• Computer Vision: Scene Analysis & Understanding

Application Category

• Search and Retrieval-Augmented AI: Web query analysis, representation and understanding
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Security and Privacy: Large-scale security measurements

Abstract

This paper aims to quantify uncertainty for SVBRDF acquisition in multi-view captures. Under uncontrolled illumination and unstructured viewpoints, there is no guarantee that the observations contain enough information to reconstruct the appearance properties of a captured object. We study this ambiguity, or uncertainty, using entropy and accelerate the analysis by using the frequency domain, rather than the domain of incoming and outgoing viewing angles. The result is a method that computes a map of uncertainty over an entire object within a millisecond. We find that the frequency model allows us to recover SVBRDF parameters with competitive performance, that the accelerated entropy computation matches results with a physically-based path tracer, and that there is a positive correlation between error and uncertainty. We then show that the uncertainty map can be applied to improve SVBRDF acquisition using capture guidance, sharing information on the surface, and using a diffusion model to inpaint uncertain regions. Our code is available at https://github.com/rubenwiersma/svbrdf_uncertainty.

Problem

• Quantify uncertainty in SVBRDF acquisition from multi-view captures
• Analyze ambiguity using entropy in frequency domain for faster computation
• Improve SVBRDF acquisition using uncertainty maps and diffusion models

Innovation

• Frequency domain analysis accelerates uncertainty computation
• Uncertainty map guides SVBRDF acquisition improvement
• Diffusion model inpaints uncertain regions effectively