Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This study addresses the challenge of stuttering classification by introducing a multiple instance neural network architecture that leverages fine-tuned wav2vec 2.0, WavLM, and Whisper encoders. By employing instance- and embedding-based multiple instance learning, the authors effectively train models on clip-level datasets to achieve both clip-level and frame-level classification of stuttering. The results indicate a significant 23% improvement in frame-level F1 score, showcasing the model's capability to extract detailed dysfluency information from less granular data.
Achieving a 23% boost in frame-level stuttering classification accuracy using clip-level data could revolutionize dysfluency assessment methods.
Stuttering detection and classification using deep learning methods has the potential to improve the process of stuttering severity assessment. Most stuttering classification datasets provide clip-level labels, making them unsuitable for fine-grained frame-level classification needed to determine the duration of individual stuttering dysfluencies. To overcome this challenge, we present a multiple instance neural network architecture based on fine-tuned wav2vec 2.0, WavLM and Whisper encoders. We apply instance- and embedding-based multiple instance learning approaches to train models on a clip-level dataset for both clip-level and frame-level stuttering classification tasks. Our results show a 23% improvement in frame-level F1 score and between 2% and 9% in clip-level F1 score, demonstrating the ability of our models to utilize clip-level data for frame-level segmentation.