HAFMat: Hybrid Priors Guided Adaptive Fusion for Single-Image Human Material Estimation

2026-06-15

AI Summary

Estimating physically based rendering (PBR) materials from a single human image is highly challenging due to the strong coupling among illumination, geometry, and reflectance in observed appearance. To address this, this work proposes HAFMat, a framework that synthesizes guidance maps by integrating multi-source heterogeneous priors—including appearance, human geometry, structural cues, and pre-trained models—and introduces a multi-level adaptive feature fusion mechanism. This mechanism dynamically combines texture-level and semantic-level cues during decoding, enabling guidance signals of varying natures to exert their influence at optimal network layers. The proposed approach effectively mitigates the inherent ambiguity in single-image material estimation, achieving state-of-the-art performance in both material reconstruction and relighting on synthetic and real-world datasets, with significantly improved accuracy and physical plausibility.

Abstract

Physically based rendering (PBR) material estimation is a fundamental appearance decomposition task with broad applications in virtual content creation, relighting, and digital human rendering. However, estimating PBR materials from a single human image remains highly ill-posed, since illumination, geometry, and reflectance are heavily entangled in the observed appearance. To mitigate this ambiguity, we propose HAFMat, a hybrid-prior-guided framework for single-image human material estimation. Our method introduces guidance maps that encode complementary cues, including appearance, body geometry, structure, and prior material predictions from pre-trained models. A key observation is that these guidance cues are heterogeneous: some cues mainly provide texture-level constraints, while others convey higher-level semantic information. To exploit this property, we design a Multi-layer Adaptive Feature Fusion Mechanism, which adaptively fuses guidance features with decoder features at different stages. This design enables texture-dominant and semantic-dominant cues to guide material decoding at appropriate levels, leading to more accurate and physically plausible material estimation. Extensive experiments on both synthetic and real data demonstrate that our method achieves state-of-the-art performance in material estimation and downstream relighting.

Problem

• PBR material estimation
• single-image human material
• appearance decomposition
• illumination-geometry-reflectance entanglement
• physically based rendering

Innovation

• Hybrid Priors
• Adaptive Feature Fusion
• Single-Image Material Estimation
• Physically Based Rendering
• Guidance Maps