Search papers, labs, and topics across Lattice.
This paper introduces GelNeuro, a fully integrated neuromorphic tactile system that combines a GelSight Mini optical tactile sensor with a Speck2f neuromorphic SoC to enable low-latency texture recognition. By capturing contact-induced marker motions as dynamic vision sensor events, the system routes data through an on-chip network to a spiking convolutional neural network classifier, achieving an impressive 96.3% accuracy on a 15-class texture recognition task. Notably, GelNeuro operates with a power consumption of only 19.6 mW, significantly outperforming traditional CPU/GPU systems in efficiency while demonstrating robust generalization across varying contact depths.
Achieving 96.3% accuracy in texture recognition with a neuromorphic system that consumes just 19.6 mW challenges the conventional power-performance trade-offs in robotic perception.
Neuromorphic visuo-tactile sensing offers a promising paradigm for low-latency and low-power robotic perception. However, existing systems still rely heavily on a host computer for event readout, preprocessing, or relaying prior to chip inference. This paper presents GelNeuro, a fully integrated sensing-computing visuo-tactile system that directly pairs a GelSight Mini-based optical tactile front end with the Speck2f neuromorphic system-on-chip (SoC). Contact-induced marker motions are captured as dynamic vision sensor (DVS) events and routed through the on-chip network to a spiking convolutional neural network (SCNN) classifier. To mitigate accuracy degradation during 8-bit deployment, a hardware-aware weight clamping strategy is introduced. Evaluated on a 15-class natural texture recognition task, hardware-in-the-loop testing on the physical chip achieves a 96.3% accuracy within an 80 ms inference window. Notably, the system consumes only 19.6 mW of board-level active power-over three orders of magnitude lower than conventional CPU/GPU baselines on the same benchmark. GelNeuro also exhibits robust generalization across unseen contact depths, demonstrating the viability of direct sensor-to-chip tactile recognition on edge neuromorphic hardware.
