Search papers, labs, and topics across Lattice.
To resolve the sensor fragmentation in monocular 3D vision, the authors develop OmniPoint, a framework that recovers metric 3D point clouds across arbitrary camera projections鈥攊ncluding pinhole, fisheye, and equirectangular鈥攚hile flexibly incorporating optional intrinsics and sparse depth priors. The architecture abandons planar depth regression in favor of decoupled ray and distance predictions, paired with a bidirectional 3D augmentation pipeline that transfers perspective labels to unannotated omnidirectional domains. As a result, the model achieves state-of-the-art zero-shot metric reconstruction across heterogeneous camera benchmarks without requiring camera-specific retraining.
Decoupling ray geometry from scene distance breaks the pervasive pinhole bias of modern 3D vision, enabling a single model to recover metric point clouds zero-shot across standard, fisheye, and 360掳 cameras.
Recovering metric 3D geometry from monocular images is a fundamental computer vision task, yet current methods remain heavily fragmented by fixed camera model assumptions and inflexible input schemes. We present OmniPoint, a unified framework designed to generalize metric reconstruction across diverse imaging sensors, including pinhole, fisheye, and equirectangular projections, while accommodating varying geometric priors. To overcome projection rigidity, OmniPoint abandons conventional planar depth regression. It instead adopts a decoupled ray and distance representation alongside a decoupled training objective, explicitly separating the camera projection model from the scene structure. To address the severe scarcity of training data for alternative cameras, we introduce a bidirectional augmentation strategy that explicitly bridges labeled perspective data and unlabeled omnidirectional domains in 3D space. Furthermore, to seamlessly integrate optional inputs like camera intrinsics or sparse depth without destabilizing the network through feature distribution shifts, we propose a robust information injection mechanism. This mechanism utilizes learnable input state embeddings to resolve architectural ambiguity and applies vectorized Gaussian smoothing to densify irregular measurements. Extensive experiments demonstrate that OmniPoint achieves state-of-the-art zero-shot performance across multiple benchmarks, establishing a robust new standard for unified monocular 3D reconstruction.
