Search papers, labs, and topics across Lattice.
This paper addresses the limitations of automatic speech recognition (ASR) systems in capturing non-verbal vocalizations (NVs) that convey important conversational cues. The authors implement three data-centric strategies to enhance the recognition of low-resource NVs, demonstrating that leveraging shared acoustic structures can significantly improve the detection of rare NV categories without compromising the quality of lexical transcription. Key results indicate that these methods lead to better recognition of events like cries and coughs, which are typically underrepresented in ASR datasets.
Non-verbal vocalizations can be effectively modeled in ASR, improving recognition of rare events without sacrificing lexical accuracy.
Modern automatic speech recognition (ASR) systems excel at transcribing lexical content but often omit nonverbal vocalizations (NVs), such as laughter, breaths, coughs, and cries, that carry conversational and affective information. Modeling NVs in ASR is challenging because NV annotations are sparse and highly long-tailed, with frequent categories such as breaths and laughter dominating rarer events such as cries and coughs. We study three data-centric strategies for improving low-resource NV recognition: (1) a two-stage curriculum that first maps all NV events to a generic token and then fine-tunes on target categories; (2) inter-token transfer from high-resource events, such as laughter and breath, to rare events, such as crying; and (3) voice-conversion augmentation with class balancing. Experiments show that shared acoustic structure across vocal events can be exploited to improve rare-category detection while preserving lexical ASR quality.
