Search papers, labs, and topics across Lattice.
This paper introduces ETHEREAL, the first dedicated processor for event-driven graph neural networks (EV-GNNs), designed to efficiently handle the unique demands of dynamic vision sensors (DVS). By integrating a neighbor-parallel spline-convolution engine with a split-2D/3D memory hierarchy, ETHEREAL achieves a remarkable latency of 25.6 渭s and an energy consumption of 1.6 渭J per inference on high-resolution vision tasks. These advancements enable real-time processing of sparse spatiotemporal events, significantly enhancing the feasibility of edge-vision applications.
ETHEREAL achieves sub-30 渭s latency for high-resolution vision tasks, setting a new standard for event-driven processing at the edge.
Dynamic vision sensors (DVS) are enticing candidates to reach the low-latency, sub-ms target of edge-vision applications, as they generate events with a $\mu$s-level time resolution. However, using DVS front ends also calls for novel algorithm/hardware back ends capable of efficiently handling streams of sparse spatiotemporal events. While event-driven graph neural networks (EV-GNNs) have emerged as a solution on the algorithmic side that is both accurate and efficient, there is no dedicated hardware to date capable of efficiently supporting their mixed requirements of dense-regular compute operations and sparse-irregular memory accesses. We therefore introduce ETHEREAL, the first EV-GNN processor chip, capable of bridging this gap by means of a neighbor-parallel spline-convolution engine combined with a split-2D/3D memory hierarchy that introduces a novel spatiotemporal event-caching mechanism. Measurement results demonstrate a 25.6$\mu$s latency and a 1.6$\mu$J energy per end-to-end event-wise inference on the state-of-the art DAGr-GNN workload and VGA-resolution (640x480 pixels) DSEC dataset.
