Search papers, labs, and topics across Lattice.
This paper introduces a multi-modal augmented reality (AR) framework that integrates photorealistic virtual objects into real-world railway sequences from the OSDaR23 dataset to address the lack of high-quality, annotated data for railway perception tasks. The framework uses Unreal Engine 5, LiDAR point clouds, and INS/GNSS data for accurate object placement and temporal stability, and it incorporates a segmentation-based refinement strategy to improve the realism of augmented sequences. The resulting OSDaR-AR dataset, designed to support the development of next-generation railway perception systems, is made publicly available.
Bridging the sim-to-real gap in railway perception, a new augmented reality framework generates realistic training data by seamlessly integrating virtual objects into real-world railway scenes, outperforming traditional methods.
Although deep learning has significantly advanced the perception capabilities of intelligent transportation systems, railway applications continue to suffer from a scarcity of high-quality, annotated data for safety-critical tasks like obstacle detection. While photorealistic simulators offer a solution, they often struggle with the ``sim-to-real"gap; conversely, simple image-masking techniques lack the spatio-temporal coherence required to obtain augmented single- and multi-frame scenes with the correct appearance and dimensions. This paper introduces a multi-modal augmented reality framework designed to bridge this gap by integrating photorealistic virtual objects into real-world railway sequences from the OSDaR23 dataset. Utilizing Unreal Engine 5 features, our pipeline leverages LiDAR point-clouds and INS/GNSS data to ensure accurate object placement and temporal stability across RGB frames. This paper also proposes a segmentation-based refinement strategy for INS/GNSS data to significantly improve the realism of the augmented sequences, as confirmed by the comparative study presented in the paper. Carefully designed augmented sequences are collected to produce OSDaR-AR, a public dataset designed to support the development of next-generation railway perception systems. The dataset is available at the following page: https://syndra.retis.santannapisa.it/osdarar.html
