Search papers, labs, and topics across Lattice.
To address the severe geometric distortions and coordinate mismatch when mapping 360掳 angular visual observations into Cartesian coordinates, this work releases Spheriverse, a large-scale multimodal benchmark comprising 64,400 synchronized spherical image-LiDAR pairs across diverse real-world environments. The authors develop SphereOcc, an occupancy prediction framework that addresses this cross-space representation gap by modulating Cartesian voxels with spherical range-azimuth geometry and adaptively re-querying visual features using range-height-azimuth geometric conditioning. SphereOcc achieves 13.91% mIoU and 24.65% GeoIoU, surpassing competitive baselines TPVFormer and SurroundOcc across all evaluated scenes, spatial splits, and restricted fields of view.
Angular-to-Cartesian projection distortions have long bottlenecked spherical 3D perception, but explicitly modulating voxel features with spherical range-azimuth geometry outperforms top surround-view models like SurroundOcc and TPVFormer across varied wild environments.
Spherical observations provide global visual context for 3D scene understanding. However, visual information is encoded in an angular domain, whereas the physical world is represented in Cartesian coordinates. This cross-space representation gap complicates geometric correspondence and semantic evidence aggregation. To delve into this challenge, we introduce Spheriverse, comprising $64,400$ temporally aligned spherical image-LiDAR pairs organized into 644 sequences. The dataset spans diverse scenes, illumination, and weather conditions, with fine-grained semantic classes. We further establish benchmarks for semantic occupancy prediction, semantic mapping, and 3D object detection, evaluating 30+ methods through overall and scene-wise comparisons. For dense prediction, we propose SphereOcc, an occupancy framework that couples spherical geometry modeling with semantic evidence retrieval. Cartesian-Spherical Representation Remodeling (CSRR) incorporates spherical range-azimuth geometry into Cartesian voxel features through region-wise modulation. Spherical Evidence Re-querying (SER) then conditions queries on voxel content and range-height-azimuth geometry to adaptively retrieve relevant semantic evidence from source spherical image features. SphereOcc achieves 13.91% mIoU and 24.65% GeoIoU, outperforming the respective best-performing methods, TPVFormer and SurroundOcc, by 1.70 and 2.10 percentage points. It also ranks first in both metrics across all five scenes, with consistent advantages across the evaluated spatial partitions and reduced fields of view. The established benchmark and source code will be available at https://feit-feiteng.github.io/Spheriverse.
