Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
NV-Bench is introduced as a benchmark for evaluating nonverbal vocalization (NV) synthesis in text-to-speech (TTS) systems, addressing the lack of standardized metrics and ground-truth references. The benchmark includes 1,651 multi-lingual utterances across 14 NV categories, paired with human reference audio. A dual-dimensional evaluation protocol is proposed, encompassing instruction alignment using paralinguistic character error rate (PCER) and acoustic fidelity by measuring the distributional gap to real recordings, demonstrating strong correlation with human perception.
Standardized evaluation of nonverbal vocalizations in TTS is now possible with NV-Bench, a new benchmark that treats NVs as communicative acts, not just acoustic artifacts.
While recent text-to-speech (TTS) systems increasingly integrate nonverbal vocalizations (NVs), their evaluations lack standardized metrics and reliable ground-truth references. To bridge this gap, we propose NV-Bench, the first benchmark grounded in a functional taxonomy that treats NVs as communicative acts rather than acoustic artifacts. NV-Bench comprises 1,651 multi-lingual, in-the-wild utterances with paired human reference audio, balanced across 14 NV categories. We introduce a dual-dimensional evaluation protocol: (1) Instruction Alignment, utilizing the proposed paralinguistic character error rate (PCER) to assess controllability, (2) Acoustic Fidelity, measuring the distributional gap to real recordings to assess acoustic realism. We evaluate diverse TTS models and develop two baselines. Experimental results demonstrate a strong correlation between our objective metrics and human perception, establishing NV-Bench as a standardized evaluation framework.