Search papers, labs, and topics across Lattice.
This study explores the application of transfer learning and the TA3N domain adaptation method to enhance sign language recognition, focusing on American Sign Language (ASL). By employing a Temporal Relational Network (TRN) to align multi-scale temporal relations, the research demonstrates that domain adaptation significantly outperforms traditional neural network-based transfer learning approaches. Notably, the findings reveal that aligning shorter-term temporal features between source and target domains is particularly effective, with RGB input consistently outperforming Optical Flow in recognition tasks.
Domain adaptation can dramatically enhance sign language recognition, achieving superior results over conventional transfer learning methods.
Sign language serves as a vital means of communication for individuals with hearing impairments, yet recognition resources for the over 100 distinct sign languages are severely lacking. In response, we present our work on sign language recognition using transfer learning and the domain adaptation method TA3N, which utilizes the Temporal Relational Network (TRN) module for aligning multi-scale temporal relations. Our findings highlight the superior performance of Domain Adaptation to neural network-based transfer learning, particularly in improving recognition of American Sign Language (ASL). Our research also identifies the effectiveness of aligning shorter-term temporal features between source and target domains. In addition to using RGB, we conducted experiments using Optical Flow mode for the sign language samples, ultimately determining that RGB outperforms Optical Flow in the majority of cases. Our work aims to improve accessibility and communication for individuals who rely on sign language as their primary mode of communication.
Tsinghua AI