Search papers, labs, and topics across Lattice.
This paper introduces FLEET, a novel feature extraction method designed for event camera data in reinforcement learning, which processes event sequences directly without aggregating them into grids. By utilizing random Fourier features and cross-attention, FLEET generates fixed-size latent representations that decouple inference costs from sensor resolution, allowing for more efficient end-to-end learning. The experimental results show that FLEET outperforms state-of-the-art methods and is more robust to variations in observation frequencies, highlighting its potential for real-time applications.
FLEET achieves superior performance in reinforcement learning with event cameras by directly processing event sequences, challenging the limitations of traditional CNN-based approaches.
Event cameras generate asynchronous, high-frequency data streams offering spatially sparse information at lower latency than traditional cameras.In principle, these properties should be ideal for the design of control policies.However, reinforcement learning research in this field remains limited as existing approaches fail to fully exploit the sensor's properties.CNN-based methods negate the sensors benefits by aggregating events into sparse grids. This couples compute cost to sensor resolution and blurs the temporal information. Meanwhile, existing generative baselines rely on the availability of trajectory data to pretrain the model. We propose FLEET (Feature Learning from Events via Efficient Tokenization), a feature extractor that processes event sequences directly. Leveraging random Fourier features and cross-attention, our architecture compresses variable streams into fixed-size latent representations. This decouples inference cost of the feature extractor's backbone from the sensor's resolution, enabling end-to-end learning without auxiliary losses. We validate FLEET on a new, high-throughput benchmark. The results demonstrate that our sequence-based approach surpasses SOTA performance and exhibits superior robustness to variations in observation frequencies.
