Search papers, labs, and topics across Lattice.
This paper introduces a novel differentiable path tracing method that incorporates polarization cues to enhance physically based inverse rendering tasks such as 3D reconstruction and reflectance estimation. By addressing the challenges posed by the rank-deficient nature of polarimetric operators, the authors develop a robust framework that combines path replay and local caching to achieve stable and unbiased gradient estimation. The key result demonstrates that this approach significantly improves the optimization of material and lighting parameters in complex scenes, thereby expanding the utility of differentiable rendering techniques.
Polarization-aware differentiable rendering can unlock new dimensions in 3D scene reconstruction by leveraging previously discarded polarization cues.
Physically based differentiable rendering has proven to be a powerful tool for inverse rendering problems (e.g., 3D reconstruction, reflectance estimation, lighting estimation). However, most existing methods operate solely on radiometric intensity, discarding valuable polarization cues that constrain scene geometry and material properties. While forward simulation of polarized light is well-defined via Mueller-Stokes calculus, extending reverse-mode differentiation to this domain presents significant challenges. The rank-deficient nature of common polarimetric operators, such as linear polarizers and diffuse reflections, violates the invertibility assumptions of standard gradient estimators like path replay backpropagation and results in numerical instability. We address this by proposing a robust, polarization-aware differentiable path tracing method. Our approach estimates unbiased gradients through a combination of path replay and local caching. This formulation enables efficient and stable optimization of material and lighting parameters in complex scenes, broadening the applicability of physically based inverse rendering. Project page: https://vcai.mpi-inf.mpg.de/projects/DPPT/
