Search papers, labs, and topics across Lattice.
This study investigates the role of prosody in sarcasm perception using a novel framework that employs neural text-to-speech (TTS) with prompt-based prosodic conditioning to manipulate acoustic dimensions like speech rate, pitch variation, and loudness. By constructing an orthogonal stimulus set, the researchers were able to isolate the independent effects of these prosodic cues, revealing that human listeners primarily rely on loudness to detect sarcasm, while a foundation model emphasizes speech rate. The findings highlight significant discrepancies in cue-weighting between human perception and model predictions, underscoring the complexities of sarcasm interpretation in synthetic speech.
Loudness is the key driver of sarcasm perception for humans, while models misprioritize speech rate, revealing a critical gap in understanding synthetic speech.
Prosody plays a central role in sarcasm perception, yet previous studies have relied on naturally produced speech that lacks fine-grained control over individual acoustic dimensions. As prosodic cues co-vary in natural data, isolating their independent contributions remains challenging. We introduce a controlled framework using neural text-to-speech (TTS) with prompt-based prosodic conditioning to manipulate speech rate, pitch variation, and loudness. An orthogonal stimulus set was constructed to enable causal testing of prosodic cue effects. Human listeners rated sarcasm and naturalness, and their judgments were compared with predictions from a foundation model capable of processing audio input. Results show that loudness primarily drives human sarcasm perception, whereas the model assigns greater weight to speech rate, leading to distinct cue-weighting patterns. This study shows how controllable neural TTS enables investigation of prosodic cue weighting in speech perception.
