Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces DualDiff3D, a novel approach that employs dual diffusion priors and a Structure-Appearance Attention (SAA) module to enhance the quality of 3D Gaussian Splatting (3DGS) under limited input views. By separating the extraction of structural information and appearance consistency into two distinct diffusion branches, the method effectively mitigates blurring artifacts caused by conflicting properties in rendered novel views. Experimental results show that DualDiff3D significantly outperforms existing state-of-the-art techniques, particularly in scenarios with flawed 3D representations, demonstrating its robustness and reliability in 3D reconstruction tasks.
DualDiff3D achieves unprecedented accuracy in 3D reconstruction by effectively disentangling structure and appearance, even from low-quality inputs.
While 3D Gaussian Splatting (3DGS) has revolutionized 3D reconstruction and novel-view synthesis, scenarios with limited input views often lead to poor reconstruction quality and artifacts in rendered novel views. Recent efforts attempt to utilize powerful diffusion priors, yet they typically process rendered and reference views concatenated along an additional dimension in a single network. These methods overlook an inherent nature that different views should maintain appearance similarity but differ in structure due to view shifts, leading to blur caused by conflicts between the two properties. In this paper, we propose DualDiff, a novel pipeline that leverages dual diffusion priors with a Structure-Appearance Attention (SAA) module to introduce reference guidance for refining low-quality novel views rendered from flawed 3D representations. Specifically, we retain one diffusion branch to focus on extracting structural information from the low-quality novel views, while introducing another branch to ensure appearance consistency with reference views. Furthermore, we present a 3D reconstruction framework named DualDiff3D, which integrates a reliability-enhanced Render-Refine-Optimize (RRO) loop to progressively and robustly incorporate the refined novel views, yielding more accurate 3DGS. Extensive experiments demonstrate that our approach outperforms state-of-the-art methods even in the inference-only setting, with further performance gains achievable through training. Our code and pre-trained weights are available at https://github.com/Akaneqwq/DualDiff3D.