Search papers, labs, and topics across Lattice.
This paper introduces a toolkit to detect spurious correlations in speech datasets by analyzing non-speech regions. The toolkit identifies if target class information can be extracted from non-speech audio, indicating a potentially misleading correlation between recording characteristics and the target. The authors demonstrate the toolkit's utility in flagging datasets where performance is likely overestimated due to these spurious correlations.
Discover hidden biases in your speech datasets: this toolkit uses non-speech audio to reveal spurious correlations that inflate performance metrics.
We introduce a toolkit for uncovering spurious correlations between recording characteristics and target class in speech datasets. Spurious correlations may arise due to heterogeneous recording conditions, a common scenario for health-related datasets. When present both in the training and test data, these correlations result in an overestimation of the system performance -- a dangerous situation, specially in high-stakes application where systems are required to satisfy minimum performance requirements. Our toolkit implements a diagnostic method based on the detection of the target class using only the non-speech regions in the audio. Better than chance performance at this task indicates that information about the target class can be extracted from the non-speech regions, flagging the presence of spurious correlations. The toolkit is publicly available for research use.
