Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces the Joint Event-RGB Adaptive Tracking (JEAT) framework, which integrates asynchronous event streams with RGB camera data to enhance vehicle tracking and traffic perception in low-light highway environments. By employing an Adaptive Extended Kalman Filter, JEAT dynamically adjusts the fusion of these modalities based on real-time measurement noise, significantly improving performance in challenging nighttime conditions. The authors also present the SEHN dataset, a synthetic resource designed to support research in event-based perception across varied environmental scenarios, addressing a critical gap in available data for this domain.
Nighttime highway perception can be revolutionized by a framework that intelligently fuses event data and RGB imagery, overcoming the limitations of traditional tracking methods.
Intelligent Transportation Systems deployed on highways predominantly rely on conventional RGB cameras for traffic perception and vehicle tracking. However, highway environments present unique challenges: the absence of artificial lighting infrastructure, combined with high vehicle velocities, results in severely degraded perception performance under low-light conditions. Specifically, nighttime scenarios suffer from motion blur, insufficient exposure, and poor signal-to-noise ratios, which catastrophically impair the reliability of RGB-based sensing systems. To address these limitations, we propose a novel Joint Event-RGB Adaptive Tracking (JEAT) framework. Unlike existing multi-sensor trackers constrained by rigid, hard-coded prioritization, JEAT merges asynchronous event streams and RGB frames into a unified joint data association optimization. By employing an Adaptive Extended Kalman Filter to continuously estimate measurement noise via NIS statistics, the framework dynamically weights and fuses both modalities, optimally harnessing event streams during dark or high-speed motion while leveraging RGB frames under bright or static conditions. Furthermore, given the absence of publicly available datasets tailored for event-based highway perception with diverse environmental conditions, we present SEHN, a large-scale synthetic dataset generated using the CARLA simulator. Our dataset encompasses diverse environmental conditions (daytime, nighttime, nighttime with out artificial lighting) and varying traffic densities, providing synchronized RGB imagery and event streams to facilitate multi-modal fusion research. Our code and datasets will be available at https://github.com/haidongwang96/SEHN.