Search papers, labs, and topics across Lattice.
This paper introduces Pano360, a novel panorama stitching approach that leverages 3D photogrammetric space to achieve geometric consistency across multiple views, addressing the limitations of traditional 2D feature-based methods. A transformer-based architecture is employed to achieve 3D awareness and aggregate global information, guiding image warping based on camera poses for global alignment. The method is trained and evaluated on a newly constructed large-scale dataset of real-world scenes, demonstrating significant improvements in alignment accuracy and perceptual quality compared to existing techniques.
Ditch the 2D stitching struggles: Pano360 warps images directly in 3D space using a transformer, creating panoramas with far better alignment and less distortion.
Prior panorama stitching approaches heavily rely on pairwise feature correspondences and are unable to leverage geometric consistency across multiple views. This leads to severe distortion and misalignment, especially in challenging scenes with weak textures, large parallax, and repetitive patterns. Given that multi-view geometric correspondences can be directly constructed in 3D space, making them more accurate and globally consistent, we extend the 2D alignment task to the 3D photogrammetric space. We adopt a novel transformer-based architecture to achieve 3D awareness and aggregate global information across all views. It directly utilizes camera poses to guide image warping for global alignment in 3D space and employs a multi-feature joint optimization strategy to compute the seams. Additionally, to establish an evaluation benchmark and train our network, we constructed a large-scale dataset of real-world scenes. Extensive experiments show that our method significantly outperforms existing alternatives in alignment accuracy and perceptual quality.
