Search papers, labs, and topics across Lattice.
PathBench is introduced as a unified benchmark for pathological speech assessment, addressing the fragmentation of research across private datasets. The benchmark compares reference-free, reference-text, and reference-audio methods across three protocols on six datasets. They also introduce Dual-ASR Articulatory Precision (DArtP), a novel reference-free metric that achieves state-of-the-art correlation with human intelligibility scores.
A new benchmark, PathBench, finally allows for standardized comparison of pathological speech assessment methods, revealing that the proposed Dual-ASR Articulatory Precision (DArtP) metric outperforms existing reference-free approaches.
Automatic speech intelligibility assessment is crucial for monitoring speech disorders and therapy efficacy. However, existing methods are difficult to compare: research is fragmented across private datasets with inconsistent protocols. We introduce PathBench, a unified benchmark for pathological speech assessment using public datasets. We compare reference-free, reference-text, and reference-audio methods across three protocols (Matched Content, Extended, and Full) representing how a linguist (controlled stimuli) versus machine learning specialist (maximum data) would approach the same data. We establish benchmark baselines across six datasets, enabling systematic evaluation of future methodological advances, and introduce Dual-ASR Articulatory Precision (DArtP), achieving the highest average correlation among reference-free methods.
