Search papers, labs, and topics across Lattice.
This paper introduces InvSplat, a feed-forward multi-view reconstruction framework for inverse rendering that predicts a structured 3D Gaussian representation with intrinsic material properties. By integrating a material estimation network with a multi-view 3D reconstruction backbone, the model enables joint prediction of geometry and reflectance parameters in a single forward pass, addressing issues of multi-view inconsistencies and lack of explicit 3D representation in existing methods. Experimental results show that InvSplat achieves improved multi-view consistency, accurate material recovery, and stable novel view rendering, making it a significant advancement over traditional RGB-based approaches.
InvSplat achieves unprecedented multi-view consistency and material recovery by predicting a structured 3D Gaussian representation in a single forward pass.
Inverse rendering aims to recover both 3D geometry and physically meaningful material properties from images, enabling applications such as relighting and novel view synthesis. Optimization-based methods achieve high fidelity but require costly per-scene fitting, while image-space learning-based approaches often suffer from multi-view inconsistencies and lack an explicit 3D representation for stable novel view rendering. We present a feed-forward multi-view reconstruction framework for inverse rendering that directly predicts a structured 3D Gaussian representation with intrinsic material attributes. Each Gaussian primitive is parameterized by mean, normal, opacity, rotation, scale, albedo, metallic, and roughness, enabling a disentangled and physically grounded scene representation. Our model integrates priors from a material estimation network with a multi-view 3D reconstruction backbone, allowing joint prediction of geometry and reflectance parameters in a single forward pass. Experiments on synthetic and real-world datasets demonstrate improved multi-view consistency compared to 2D baselines, accurate material recovery, and stable novel view rendering. Our representation further supports physically-based relighting and more faithful modeling of view-dependent effects compared to existing RGB-based feed-forward reconstruction methods. Our project webpage is: $\href{https://poliik.github.io/invsplat/}{\text{https://poliik.github.io/invsplat/}}$.
