Search papers, labs, and topics across Lattice.
This study evaluates the impact of frame rate and neural network architecture on the automated detection of autism-related self-stimulatory behaviors from video data. By comparing long short-term memory (LSTM) and gated recurrent unit (GRU) models at various frame sampling intervals, the authors achieved peak accuracies of 97.5% and 98.75%, respectively, at a 15-frame interval, significantly outperforming previous convolutional neural network baselines. Additionally, the research identifies effective data augmentation strategies, revealing that horizontal flip and careful management of upsampling are critical for enhancing model performance in small datasets.
Achieving up to 98.75% accuracy in detecting autism-related behaviors highlights the potential of sequence-based models over traditional CNNs in data-scarce environments.
Autism spectrum disorder (ASD) affects over 75 million individuals worldwide, yet scalable computational methods for remote behavioral screening remain limited. This study addresses two complementary challenges in automated detection of autism-related self-stimulatory behaviors from video: (1) identifying the optimal sequence-based neural network architecture and temporal sampling rate, and (2) characterizing data augmentation strategies for training on small behavioral datasets. For the first objective, long short-term memory (LSTM) and gated recurrent unit (GRU) models were trained on pose-derived features from the Self-Stimulatory Behavior Diagnosis (SSBD) dataset at frame sampling intervals of 1, 5, 15, 30, 45, and 90 frames. Both architectures exceeded prior convolutional neural network (CNN) baselines (62-76% accuracy), with peak accuracies of 97.5% (LSTM) and 98.75% (GRU) at a sampling interval of every 15 frames. For the second objective, ten data augmentation strategies were applied to an I3D transfer learning pipeline, with an ablation study quantifying the marginal contribution of each technique. Horizontal flip achieved the highest standalone accuracy (48.78%), while exclusion of upsampling from the augmentation pipeline produced the largest performance degradation, indicating its necessity for complex behavioral video augmentation. A personalized machine learning approach, in which per-subject models were trained and tested on temporally split segments of each video, produced consistent predictions (mean loss 1.84, SD 0.79). These results provide practitioners with concrete guidance on architecture selection, sampling rate, and augmentation strategy for video-based behavioral classification in data-scarce clinical domains.
CMU ML