Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces a lightweight hybrid reservoir computing (HRC) model for isolated sign language video recognition, addressing the high computational costs associated with deep learning approaches. By utilizing MediaPipe to extract keypoints and combining deep and bidirectional reservoir computing, the model effectively captures the spatial and temporal dynamics of gestures. The HRC method achieved competitive accuracies of 61.12% for Top-1 and 92.56% for Top-10 on the WLASL100 dataset, while significantly reducing training time, making it suitable for edge device deployment.
Achieving 92.56% accuracy on sign language recognition with a model that trains in seconds opens new avenues for real-time applications on edge devices.
Sign language recognition (SLR) enhances communication between hearing and hearing-impaired individuals. Although deep learning (DL) has achieved promising performance in SLR, its high computational cost limits deployment on edge devices. To address this challenge, we propose a lightweight reservoir computing (RC)-based approach for SLR. In the proposed method, MediaPipe extracts body and hand keypoints to capture the spatial and temporal dynamics of gestures. These keypoints are then processed by a hybrid reservoir computing (HRC) architecture that combines deep reservoir computing (DRC) and bidirectional reservoir computing (BRC), transforming the input into a high-dimensional dynamic representation. A ridge regression model maps the final HRC state to class labels. This HRC-based SLR method achieved Top-1, Top-5, and Top-10 accuracies of 61.12%, 86.05%, and 92.56%, respectively, on the Word-Level American Sign Language 100 (WLASL100) video dataset, demonstrating competitive performance compared to deep learning-based approaches. Additionally, due to the lightweight nature of RC, the training time was drastically reduced to only a few seconds compared with DL-based methods such as Bi-GRU.This method offers low computational cost, showing its potential for deployment on edge devices.