Search papers, labs, and topics across Lattice.
The paper introduces NeuMatEx, a differentiable inverse rendering method that efficiently extracts spatially varying neural materials from multi-view images. By leveraging a Large Material Reconstruction Model (LMRM) to predict initial base color, neural material latents, and uncertainty guides, the method overcomes the challenges posed by the nonlinear structure of neural material latent spaces. Experimental results demonstrate that NeuMatEx achieves superior visual quality and material decomposition compared to traditional Physically Based Rendering (PBR) methods, enhancing the realism of rendered materials.
NeuMatEx outperforms PBR techniques by extracting complex neural materials with unprecedented visual fidelity and precision from multi-view images.
Neural materials can represent complex specular reflections and scattering effects in a compact, universal basis. However, acquiring and authoring such materials remains challenging. We present NeuMatEx, a differentiable inverse rendering method for extracting spatially varying neural materials from images. The nonlinear structure of neural material latent spaces makes optimization with naive inverse rendering infeasible. To address this, we train a Large Material Reconstruction Model (LMRM) that directly predicts initialbase color, neural material latents, and aleatoric uncertainty guides from images. This material prior provides a good initialization and better constrains our subsequent optimization using inverse path tracing. The predicted uncertainty further helps by anchoring high-confidence regions more tightly to the LMRM prediction, preventing lighting and complex specular effects from being baked into materials. Experiments on synthetic and real assets show that NeuMatEx extracts complex materials with better visual quality and material decomposition than PBR-based methods.
NVIDIA