Search papers, labs, and topics across Lattice.
This paper introduces UniSHARP, an extension of the SHARP view synthesis method designed for universal monocular rendering across various camera systems, including conventional and wide-field-of-view configurations. By aligning images in a unified omnidirectional latent space and employing implicit alignment in feature and Gaussian spaces, UniSHARP effectively generates photorealistic views. Extensive evaluations on a newly constructed benchmark reveal that UniSHARP significantly outperforms existing methods in rendering quality across diverse imaging scenarios.
UniSHARP achieves unprecedented photorealistic view synthesis across a continuum of camera systems, outperforming traditional methods by a substantial margin.
In this work, we focus on extending SHARP, the popular photorealistic view synthesis method, for universal monocular rendering across a continuum of camera systems, from conventional perspective cameras to wide-field-of-view, fisheye and omnidirectional panoramic settings. To overcome the pinhole-specific assumptions of SHARP, our key idea is to align various images in a unified omnidirectional latent space. Thus, we propose UniSHARP, which performs implicit alignment in both feature and Gaussian spaces. Specifically, Gaussian primitives are arranged along rays and radial distances in a ray-based universal representation, while 2D semantic and 3D spatial features extracted from UniK3D-inspired encoders are jointly decoded to generate the complete Gaussian cloud. To comprehensively evaluate our method, we construct a benchmark covering diverse imaging systems across various scenes. The benchmark is further stratified by field of view (FoV) to enable fine-grained assessment of the universal monocular rendering task. Extensive experiments on the proposed benchmark demonstrate the effectiveness of UniSHARP, outperforming alternative methods by a large margin. The project page can be found at: https://insta360-research-team.github.io/Unisharp-website/
