Search papers, labs, and topics across Lattice.
The paper introduces NeRFscopy, a self-supervised pipeline for dynamic 3D reconstruction of deformable endoscopic tissues from monocular video. It addresses challenges like tissue deformation and unknown camera trajectories by employing a canonical radiance field coupled with a time-dependent deformation field parameterized by SE(3) transformations. The method demonstrates accurate novel view synthesis on challenging endoscopy scenes, outperforming existing approaches without relying on templates or pre-trained models.
Reconstruct dynamic 3D models of squishy internal organs from standard endoscopic video, no pre-training required.
Endoscopy is essential in medical imaging, used for diagnosis, prognosis and treatment. Developing a robust dynamic 3D reconstruction pipeline for endoscopic videos could enhance visualization, improve diagnostic accuracy, aid in treatment planning, and guide surgery procedures. However, challenges arise due to the deformable nature of the tissues, the use of monocular cameras, illumination changes, occlusions and unknown camera trajectories. Inspired by neural rendering, we introduce NeRFscopy, a self-supervised pipeline for novel view synthesis and 3D reconstruction of deformable endoscopic tissues from a monocular video. NeRFscopy includes a deformable model with a canonical radiance field and a time-dependent deformation field parameterized by SE(3) transformations. In addition, the color images are efficiently exploited by introducing sophisticated terms to learn a 3D implicit model without assuming any template or pre-trained model, solely from data. NeRFscopy achieves accurate results in terms of novel view synthesis, outperforming competing methods across various challenging endoscopy scenes.
