Search papers, labs, and topics across Lattice.
This paper introduces a novel inverse rendering method that leverages active RGB-NIR imaging to achieve robustness against varying ambient illumination. The key idea is to use NIR flash illumination to obtain stable point-light shading, which is then combined with multi-view RGB images in a three-stage inverse rendering pipeline to reconstruct geometry and reflectance. Experiments on a newly collected multi-view RGB-NIR dataset demonstrate that the proposed method outperforms existing approaches in terms of accuracy and robustness to ambient lighting.
NIR imaging cuts through ambient light to enable surprisingly robust 3D reconstruction.
Inverse rendering aims to reconstruct geometry and reflectance of objects from images. Despite recent progress, existing methods often produces inaccurate reconstructions that are sensitive to ambient illumination conditions. Here we introduce an ambient-robust inverse rendering method enabled by active RGB-NIR imaging. Our key insight is to leverage near-infrared (NIR) flash illumination-imperceptible to human observers-to obtain stable point-light shading that is largely invariant to ambient illumination. By using multi-view RGB images illuminated by ambient light and NIR images acquired with active NIR flash illumination, we reconstruct accurate geometry and reflectance by exploiting the complementary benefits of RGB and NIR images via a three-stage inverse rendering method. To enable dense multi-view acquisition, we develop an active imaging system equipped with a RGB-NIR camera and a NIR flash mounted on a mobile base. Using this system, we collect the first multi-view RGB-NIR inverse rendering dataset captured under multiple ambient illumination conditions. Experiments demonstrate that our method outperforms prior approaches, achieving accurate geometry and reflectance estimation across multiple ambient lighting scenarios.
