From Phase to Phenomenon: Self-Supervised Learning of Subsurface Scattering with Minimal Phase-shift Inputs

2026-06-28

AI Summary

This study addresses the challenge of efficiently learning subsurface scattering (SSS) light transport representations from extremely limited input imagery to enable high-fidelity reconstruction of objects with unseen complex geometry and materials. The authors propose a self-supervised pretraining framework that, in a multi-view, multi-object setting, trains an encoder using only eight high-frequency phase-shifting profilometry (PSP) images per view to learn transferable SSS representations, which are then combined with a decoder to reconstruct dense scattering responses. Key innovations include PSP-tailored data augmentation strategies, a dedicated scattering reconstruction loss, and kNN-based classification evaluation. The method substantially outperforms existing approaches in spatially varying relighting and representation tasks, reduces image requirements by several orders of magnitude, and supports reconstruction of anisotropic scattering effects.

Abstract

We propose a self-supervised pretraining framework for learning sub-surface scattering (SSS) light transport representations from minimal input. Our method leverages a stereo projector-camera setup that captures only eight high-frequency phase-shift profilometry (PSP) images per view to pretrain an encoder in a multi-view, multi-object setting. We introduce a tailored augmentation strategy for PSP-based SSS data, and show that it significantly outperforms standard ImageNet-style augmentations for SSL pretraining. The pretrained encoder learns generalizable SSS representations that transfer effectively to downstream tasks, including spatially varying relighting and representation evaluation using a kNN classifier. Combined with a decoder, the model reconstructs dense scattering footprint responses, trained using a dedicated cost function that improves accuracy, particularly for anisotropic footprints. Despite using only eight input images per view, our approach generalizes to unseen objects with complex geometry and material properties, achieving high-fidelity reconstructions while requiring orders of magnitude fewer images than prior methods.

Problem

• subsurface scattering
• self-supervised learning
• phase-shift profilometry
• light transport
• few-shot reconstruction

Innovation

• self-supervised learning
• subsurface scattering
• phase-shift profilometry
• light transport
• data augmentation