Search papers, labs, and topics across Lattice.
SplatGuide introduces a novel approach for pose-free novel view synthesis by integrating 3D geometric understanding with multi-view diffusion, effectively closing the information disconnect in existing pipelines. The method leverages a single 3D Gaussian scene to simultaneously provide pixel-aligned geometric conditioning, occlusion-aware reference selection through per-Gaussian visibility, and feature-level guidance via reconstruction tokens. This innovative strategy leads to state-of-the-art performance across multiple datasets, notably exceeding ground-truth-pose baselines on RealEstate10K with a limited number of input views.
Achieving state-of-the-art pose-free novel view synthesis by reusing a single 3D scene across multiple complementary roles transforms how we approach 3D reconstruction and rendering.
Generating photorealistic novel views from unposed images requires both 3D geometric understanding and the ability to synthesize unseen content. A natural strategy combines feed-forward 3DGS reconstruction with multi-view diffusion. Yet prior pipelines extract at most one signal from the reconstruction, either pixel rendering or learned features, while none exploits per-Gaussian visibility for occlusion-aware reference selection. This *information disconnect* leaves renderable geometry, visibility cues, and learned features unused. SplatGuide closes this disconnect by reusing a single 3DGS scene across three complementary roles. Rendered images provide pixel-aligned geometric conditioning. Per-Gaussian source-view indices are rendered into a target-view voting map for occlusion-aware reference selection. Reconstruction tokens supply feature-level guidance via cross-attention. All three signals derive from the same reconstruction forward pass. Across RealEstate10K, DL3DV, Tanks-and-Temples, and Mip-NeRF 360, SplatGuide achieves state-of-the-art pose-free novel view synthesis. On RealEstate10K, with a moderate number of input views, it surpasses the ground-truth-pose baseline.
