Search papers, labs, and topics across Lattice.
This paper introduces SHERPA, a novel framework designed to enhance the generation of 360掳 panoramas by addressing the challenges posed by the equirectangular projection's topology. By integrating frequency-selective Circular RoPE and a Dual-Path Training Scheme, SHERPA effectively aligns the training of text-to-image diffusion models with the unique requirements of panoramic imagery. The key finding is that SHERPA successfully generates high-quality 360掳 panoramas that cater to both photorealistic and stylized environments, demonstrating significant improvements in coherence and style fidelity.
Seam-sensitive adaptations in SHERPA enable the generation of 360掳 panoramas that maintain both photorealism and artistic style, overcoming traditional planar training limitations.
Panoramic imagery is increasingly used in world-generation, games, and simulation, where users may need not only photorealistic scenes but also stylized and non-photorealistic environments. Large-scale text-to-image diffusion and flow models provide broad style and semantic priors for this goal, but planar image training misaligns them with the wrap-around topology and polar regions of $360^\circ$ panoramas represented in equirectangular projection (ERP). We present SHERPA, a lightweight adaptation framework that combines frequency-selective Circular RoPE, Circular Latent Encoding/Decoding, image-side FFN adapters, and a Dual-Path Training Scheme. Circular RoPE replaces only the seam-sensitive high-frequency horizontal RoPE band with integer-periodic harmonics while preserving the pretrained lower-frequency spectrum. The Paired Panorama Path supervises geometry, while the Unpaired Style Path uses self-supervised yaw consistency for target-free stylized prompts. As a result, SHERPA generates $360^\circ$ panoramas across both photorealistic panorama domains and open-domain stylized prompts.
