Search papers, labs, and topics across Lattice.
This paper introduces a novel sparse-view synthesis framework based on 3D Gaussian Splatting designed for unconstrained real-world scenarios with distractors. It enhances 3D representation from sparse, unconstrained image collections by using reference-guided view refinement with a diffusion model and a transient mask. The method also employs pseudo-view generation and sparsity-aware Gaussian replication to address sparse regions in the Gaussian field, leading to improved rendering quality.
Achieve high-fidelity 3D rendering from sparse, unconstrained real-world images by intelligently synthesizing novel views with diffusion models and Gaussian replication.
We propose a 3D novel sparse-view synthesis framework for unconstrained real-world scenarios that contain distractors. Unlike existing methods that primarily perform novel-view synthesis from a sparse set of constrained images without transient elements or leverage unconstrained dense image collections to enhance 3D representation in real-world scenarios, our method not only effectively tackles sparse unconstrained image collections, but also shows high-quality 3D rendering results. To do this, we introduce reference-guided view refinement with a diffusion model using a transient mask and a reference image to enhance the 3D representation and mitigate artifacts in rendered views. Furthermore, we address sparse regions in the Gaussian field via pseudo-view generation along with a sparsity-aware Gaussian replication strategy to amplify Gaussians in the sparse regions. Extensive experiments on publicly available datasets demonstrate that our methodology consistently outperforms existing methods (e.g., PSNR - 17.2%, SSIM - 10.8%, LPIPS - 4.0%) and provides high-fidelity 3D rendering results. This advancement paves the way for realizing unconstrained real-world scenarios without labor-intensive data acquisition. Our project page is available at $\href{https://robotic-vision-lab.github.io/SaveWildGS/}{here}$
