Search papers, labs, and topics across Lattice.
This paper introduces Vis2Reg, a visibility-aware framework for 3D-2D liver registration that addresses the challenges of occlusion and limited visibility in laparoscopic surgery. By leveraging a novel self-supervision technique that utilizes intraoperative masks to derive a 3D supervision signal, the method enhances robustness during extreme partial visibility scenarios. The results show a Dice score of 92.6% and a Chamfer Distance of 1.43 mm, indicating significant improvements in accuracy and efficiency for real-time applications in AR-guided surgery.
Visibility-aware self-supervision enables robust 3D-2D liver registration even under extreme occlusion, achieving a Dice score of 92.6%.
Accurate 3D--2D liver registration, which aligns preoperative 3D models to partial, view-dependent intraoperative surface observations, is critical for AR-guided laparoscopic surgery but remains challenging due to severe occlusion, limited visibility, and the lack of 3D ground-truth supervision. Existing landmark-free approaches perform partial-to-complete geometric alignment, yet robust self-supervision under extreme partial visibility remains difficult. We propose Vis2Reg, a visibility-aware registration framework that explicitly constrains deformation using mask-consistent visible regions. We introduce a visibility-aware self-supervision that derives a visible-domain 3D supervision signal from intraoperative masks, enabled by differentiable point rasterization and mask-guided back-projection. This formulation improves robustness under severe occlusion while maintaining fully self-supervised learning. Vis2Reg combines a robust geometric rigid initialization module with an implicit neural deformation field for stable alignment. Vis2Reg achieves a Dice score of 92.6\% and a Chamfer Distance of 1.43 mm on real intraoperative datasets, with 111 ms per-frame inference time, demonstrating both accuracy and practical efficiency.
