Search papers, labs, and topics across Lattice.
This study introduces ExtraGS, a novel framework that enhances endoscopic view extrapolation by integrating diffusion-guided 3D Gaussian Splatting with an uncertainty-guided virtual camera sampling strategy. By actively exploring blind spots and refining rendered views through a diffusion model, ExtraGS effectively generates pseudo observations that improve the synthesis of novel views while minimizing artifacts. Extensive experiments reveal that this approach significantly outperforms existing methods in endoscopic novel view synthesis, addressing critical limitations in conventional endoscopic perception.
ExtraGS slashes extrapolation artifacts in endoscopic view synthesis, setting a new benchmark for reliable surgical navigation.
Robot-assisted minimally invasive surgery (MIS) critically depends on reliable endoscopic perception for navigation and safety. However, conventional endoscopes provide only a limited field of view, leaving large portions of surrounding anatomy unobserved. Recent neural rendering approaches, such as Neural Radiance Fields and 3D Gaussian Splatting, enable novel view synthesis from endoscopic videos, but their reliance on sparse observations often leads to severe artifacts when extrapolating beyond the training trajectory.In this work, we propose ExtraGS, a framework for enhancing endoscopic view extrapolation via diffusion-guided 3D Gaussian Splatting. Starting from an initial reconstruction, we introduce an uncertainty-guided virtual camera sampling strategy to actively explore blind spots and maximize information gain. The rendered views from these sampled locations are refined using a diffusion model to recover plausible anatomical structures, producing pseudo observations that guide further optimization. To prevent the generated content from degrading reliable regions, we adopt a confidence-weighted fine-tuning strategy when incorporating these pseudo observations.Extensive experiments on multiple public endoscopic datasets demonstrate that ExtraGS significantly reduces extrapolation artifacts and achieves state-of-the-art performance in endoscopic novel view synthesis.
