Search papers, labs, and topics across Lattice.
This paper introduces a novel framework that re-tokenizes event camera data into discrete neural events, which encapsulate local spatio-temporal contexts through a learnable coding mechanism. By triggering a neural event with each code flip, the method effectively compresses the data stream while maintaining essential temporal dynamics. The results show that networks trained on these neural events achieve performance comparable to or better than existing state-of-the-art models, all while halving the event rate.
Neural events can reduce event camera data by 50% while matching or exceeding the performance of leading object detection and classification models.
Event cameras capture dynamic scenes with exceptional temporal fidelity by representing them as a continuous stream of microsecond resolution \textit{events}. Each individual event, however, only carries minimal semantic value, merely signaling a localized brightness change. To derive meaningful signals, downstream algorithms need to quickly integrate cues from a potentially massive torrent of low-information events. Current architectures, however, are easily overwhelmed, struggling to balance capturing fine-grained temporal dynamics and maintaining a manageable data throughput. This paper proposes a framework to re-tokenize event streams into a small set of highly informative \textit{neural events}, each representing a local spatio-temporal context window with a discrete learnable code. Every time this code flips, a neural event is triggered, yielding a highly compressed data stream. We demonstrate that, across object detection and classification, networks trained on neural events are on par or surpass the performance of state-of-the-art approaches while reducing the event rate by a factor of 2.0.
