Search papers, labs, and topics across Lattice.
This paper introduces CLANE, a novel continual learning system for action recognition using event cameras deployed on Intel Loihi 2 neuromorphic hardware. CLANE combines a spiking 2D CNN with a custom CLP-SNN learning head, enhanced with a Temporal Aggregation Layer and a fixed-point Normalization Layer, to process event-based action clips. Experiments on the THU E-ACT-50 dataset demonstrate that CLANE achieves 70.4% accuracy in continual learning while providing significant energy and latency improvements compared to a CNN+GRU+CLP baseline on an edge GPU.
Neuromorphic hardware can now continually learn new actions from event camera data with 100x energy reduction and 16x lower latency compared to edge GPUs.
Recognizing and continuously learning novel human actions without forgetting prior classes is a requirement for emerging AR/VR and robotics applications. For these applications, both on-device processing and learning are essential for privacy and low-latency adaptation. Event cameras address the efficiency of visual sensing with sparse, asynchronous output that is naturally compatible with neuromorphic processing. Yet no prior system has deployed a continual on-device learning pipeline for event-based action recognition using neuromorphic hardware. We present CLANE, Continual Learning of Actions on Neuromorphic Hardware from Event Cameras, deployed end-to-end on Intel Loihi 2. CLANE combines a spiking 2D CNN for spatiotemporal feature extraction with CLP-SNN as its on-chip learning head, extended to action clips via a Temporal Aggregation Layer and a fixed-point Normalization Layer, both novel Loihi 2 modules. On THU E-ACT-50, a 50-class dataset captured under real-world conditions, CLANE achieves 70.4% accuracy in a continual learning task while delivering more than 100x energy reduction and 16x lower latency over a sequential CNN+GRU+CLP edge GPU baseline, validated through iso-algorithm cross-platform benchmarking across three evaluation levels.
