Search papers, labs, and topics across Lattice.
The paper introduces InterFuserDVS, an extension of the InterFuser model that incorporates Dynamic Vision Sensors (DVS) data to improve the robustness of autonomous driving systems. A novel token-based fusion strategy is used to integrate accumulated event frames into InterFuser's transformer backbone, leveraging the complementary nature of RGB, LiDAR, and DVS data. Evaluated on the CARLA Leaderboard, InterFuserDVS achieves a Driving Score of 77.2 and a Route Completion of 100%, demonstrating improved performance in challenging conditions.
Event cameras can significantly boost the reliability of autonomous driving in high-dynamic-range and high-speed scenarios, achieving perfect route completion in CARLA benchmarks.
Autonomous driving systems rely heavily on robust sensor fusion to perceive complex envi- ronments. Traditional setups using RGB cameras and LiDAR often struggle in high-dynamic- range scenes or high-speed scenarios due to motion blur and latency. Dynamic Vision Sensors (DVS), or event cameras, offer a paradigm shift by capturing asynchronous brightness changes with microsecond temporal resolution and high dynamic range. In this paper, we propose an extended architecture of the state-of-the-art InterFuser model, integrating DVS as an additional modality to enhance perception reliability. We introduce a novel token-based fusion strategy that incorporates accumulated event frames into the transformer-based backbone of InterFuser. Our method leverages the complementary nature of RGB, LiDAR, and DVS data. We evaluate our approach on the Car Learning to Act (CARLA) Leaderboard benchmarks, demonstrating that the inclusion of DVS improves the robustness of the driving agent, achieving a competitive Driving Score of 77.2 and a superior Route Completion of 100%. The results indicate that event-based vision is a promising direction for improving safety and performance in adverse lighting and dynamic conditions.
