Search papers, labs, and topics across Lattice.
This paper introduces a convolutional neural shading (CNS) pipeline that enhances 3D shape reconstruction from multi-view images by addressing the limitations of traditional neural radiance fields, which rely on single-point geometric information. By incorporating a neural shader, the method captures detailed local geometry even in challenging areas like dark and textureless regions, leading to more accurate geometry predictions. The introduction of a fine-detail displacement network further improves surface quality by leveraging spatial correlations, resulting in significant advancements over existing state-of-the-art techniques in both shape and image quality.
Achieving unprecedented accuracy in 3D shape reconstruction, this method captures fine details even in the most challenging visual conditions.
We propose a convolutional neural shading (CNS), a novel pipeline to reconstruct high-quality 3D shapes from multi-view images. Several recent studies have used neural radiance fields and other neural differentiable rendering methods to understand 3D geometry. However, these approaches rely on single-point geometric information, such as positions and normals of the surface, leading to a lack of detailed local geometry. Our approach addresses the inherent limitations of single-point information by leveraging a neural shader to capture variations even in dark and textureless regions with a convolutional neural shader, resulting in far more accurate geometry predictions. Additionally, our method mitigates surface irregularities at image boundaries by introducing a fine-detail displacement network, which utilizes spatial information of surface geometry and learns fine displacement details by correlating neighboring values in the rendering coordinates. Through extensive experiments, our proposed method has demonstrated significant quality improvements in the reconstructed shapes and rendered images over current state-of-the-art methods.
