Search papers, labs, and topics across Lattice.
This paper introduces Cyclops, a novel framework that transforms sparse Non-Repetitive Scanning LiDAR (NRS-LiDAR) intensity data into dense RGB video representations, addressing the limitations of traditional cameras in low-light and high-dynamic-range environments. By leveraging a frozen pre-trained densification module and employing Latent Bridge Matching (LBM) with a learned velocity field, Cyclops enables camera-free inference for all-day perception tasks. Experimental results reveal that the synthesized RGB outputs significantly enhance the performance of standard RGB-based perception models in tasks such as semantic segmentation and lane detection, particularly in challenging lighting conditions.
Cyclops enables LiDAR to produce high-quality RGB video, outperforming traditional cameras in low-light scenarios and revolutionizing robotic perception.
Conventionally, robotic perception relies heavily on cameras due to the rich semantic texture they provide. However, their performance degrades significantly in low-light or high-dynamic-range environments. Conversely, while Light Detection and Ranging (LiDAR) captures illumination-invariant geometric and intensity properties, the resulting data are typically single-channel and sparse, creating a significant modality gap when applying vision models pre-trained on RGB datasets. In this paper, we propose Cyclops, a framework that translates sparse Non-Repetitive Scanning LiDAR (NRS-LiDAR) intensity into RGB video, enabling camera-free inference for all-day perception tasks. Our approach first converts sparse LiDAR intensity projections into dense representations via a frozen pre-trained densification module, serving as a geometrically rich source condition. The dense intensity latent is then transported toward the target RGB distribution through Latent Bridge Matching (LBM) with a learned velocity field in a few ODE integration steps. To mitigate inter-frame flickering, we inject prior-frame context via temporal attention layers and further formulate the velocity field as a policy optimized by a differentiable terminal reward that encourages terminal fidelity through backpropagation along the ODE trajectory. Extensive experiments demonstrate that the synthesized RGB, including those generated under near-dark conditions, enable standard RGB-based perception models to substantially outperform both LiDAR baselines and conventional cameras on semantic segmentation, lane detection, and point cloud colorization across diverse lighting conditions.
