Search papers, labs, and topics across Lattice.
This paper addresses the challenge of aligning generative 3D reconstructions with partial monocular observations, a task complicated by the asymmetries between sparse inputs and dense priors. The authors introduce a training-free geometric alignment framework that utilizes a 3D similarity transformation (Sim(3)) to recover accurate scale and pose, incorporating a coarse-to-fine strategy and a Hallucination Filtering operation to mitigate outliers. Experimental results show that their approach significantly outperforms traditional geometric methods and current learning-based techniques, establishing a new benchmark for this alignment task.
Achieving stable and accurate alignment of generative 3D models with partial observations could redefine the standards for 3D reconstruction in computer vision.
Aligning generative 3D reconstructions with partial monocular observations is a critical but under-explored challenge in computer vision. This task is inherently ill-posed due to severe asymmetries between noisy, sparse monocular inputs and dense generative priors, whose scale ambiguity and geometric hallucinations, combined with the lack of initial overlap, render traditional registration pipelines ineffective. To resolve these issues, we propose a training-free and interpretable geometric alignment framework that grounds generative 3D priors via a 3D similarity transformation (Sim(3)), which can recover accurate metric scale and pose. Specifically, we introduce an explicit scale factor to resolve metric ambiguity and employ a coarse-to-fine alignment strategy, leveraging geometry-aware descriptors for robust initialization and a decoupled closed-form solver for precision refinement. In addition, we introduce a Hallucination Filtering operation to effectively suppress outliers caused by hallucinated geometry. To evaluate alignment performance under these extreme conditions, we introduce GenPMOAlign--Where2Place, a rigorous benchmark specifically designed for Generative-to-Partial Monocular Observational Alignment. Experiments demonstrate that our method achieves stable and accurate registration, substantially outperforming both classical geometric pipelines and state-of-the-art learning-based baselines. Code and the benchmark will be publicly released.
