Search papers, labs, and topics across Lattice.
This paper introduces HAFMat, a hybrid-prior-guided framework that enhances single-image human material estimation by leveraging guidance maps that incorporate diverse cues such as appearance, body geometry, and prior material predictions. The proposed Multi-layer Adaptive Feature Fusion Mechanism intelligently fuses these heterogeneous cues at different stages of the decoding process, allowing for improved texture and semantic guidance. Experimental results show that HAFMat outperforms existing methods in both material estimation accuracy and downstream relighting tasks, highlighting its effectiveness in addressing the challenges of entangled illumination, geometry, and reflectance.
By intelligently fusing heterogeneous guidance cues, HAFMat achieves unprecedented accuracy in estimating human materials from a single image.
Physically based rendering (PBR) material estimation is a fundamental appearance decomposition task with broad applications in virtual content creation, relighting, and digital human rendering. However, estimating PBR materials from a single human image remains highly ill-posed, since illumination, geometry, and reflectance are heavily entangled in the observed appearance. To mitigate this ambiguity, we propose HAFMat, a hybrid-prior-guided framework for single-image human material estimation. Our method introduces guidance maps that encode complementary cues, including appearance, body geometry, structure, and prior material predictions from pre-trained models. A key observation is that these guidance cues are heterogeneous: some cues mainly provide texture-level constraints, while others convey higher-level semantic information. To exploit this property, we design a Multi-layer Adaptive Feature Fusion Mechanism, which adaptively fuses guidance features with decoder features at different stages. This design enables texture-dominant and semantic-dominant cues to guide material decoding at appropriate levels, leading to more accurate and physically plausible material estimation. Extensive experiments on both synthetic and real data demonstrate that our method achieves state-of-the-art performance in material estimation and downstream relighting.
