Search papers, labs, and topics across Lattice.
This research explores the use of depth image-based point cloud data for sign language recognition, addressing the scarcity of depth datasets by generating synthetic depth images from RGB inputs using the Depth Anything V2 network. By evaluating classification accuracies across three sign language datasets with various PointNet architectures, the study reveals that while original depth-based models generally outperform synthetic ones, certain synthetic models exceed original performance in specific cases. These findings highlight the potential of synthetic data in enhancing sign language recognition systems, suggesting a nuanced relationship between data type and model efficacy.
Original depth images typically outperform synthetic ones in sign language recognition, but surprising instances show the opposite, challenging assumptions about data quality.
Research regarding the sign language recognition mostly relies on RGB images, whileas sign language datasets that provide depth images are limited. Point clouds obtained from depth images can be used for sign language recognition with neural networks like PointNet. In recent years, various neural networks are used for generating realistic depth images from monocular RGB images. In this work, synthetic depth images were created from RGB images using Depth Anything V2 network. For this purpose, three sign language datasets (Real-time ASL Fingerspelling, KArSL, AUTSL) which contain both RGB and depth images were used. Classification accuracies of the point cloud data created from both original and synthetic depth images using various PointNet architectures were measured for sign language recognition. From the original and synthetic point clouds, frame based, Point Gesture Map and Long Short Term Memory data models were used for classification and their performances were compared. In the results, both original and synthetic based data achieved acceptable performance in most models. In general, original depth based point cloud models performed better than synthetic ones, however in some models synthetic depth based models performed better than the originals.
