Search papers, labs, and topics across Lattice.
This paper introduces JSL-DC, the largest dataset for Japanese Sign Language (JSL) consisting of 36.7K videos from 19 Deaf signers, aimed at enhancing sign language acquisition for deaf children. The dataset was meticulously curated with input from Deaf and Coda linguists to ensure relevance for parent-child communication, and it includes linguist-derived descriptions to help distinguish confusable signs. A model leveraging these descriptions achieved a 9.8% improvement over existing state-of-the-art recognition methods on challenging sign pairs, highlighting the dataset's potential to advance SL recognition technology.
A novel dataset and model that significantly improve JSL sign recognition by addressing confusable signs, crucial for effective communication between deaf children and their hearing parents.
Effective sign language (SL) acquisition is crucial for deaf children, yet 95% are born to hearing parents who often lack proficiency in SL. SL recognition can power learning tools to help parents communicate with their children. However, Japanese Sign Language (JSL) lacks large-scale, multi-signer datasets, hindering the development of models that can generalize to new users. To address this gap, we introduce JSL-DC, the largest JSL dataset by video count, comprising 36.7K videos from 19 signers. The entire process was Deaf-centric: the lexicon comprising 270 JSL words was selected by Deaf and Coda linguists to facilitate parent-child communication, all participants were Deaf individuals who use JSL daily, and the data underwent a two-stage review process involving Deaf linguists. Moreover, we provide linguist-derived descriptions for distinguishing confusable signs. We demonstrate that the proposed model inspired by the descriptions outperforms state-of-the-art recognition methods by 9.8% on the confusable subset. The dataset, along with its linguistic description that inspires new models, will be released under a CC-BY 4.0 license to accelerate research in SL recognition.
