Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This study investigates the interplay between text- and audio-based features in understanding emotional valence and arousal in spontaneous Finnish speech. By utilizing a newly released affective speech corpus, the authors demonstrate that combining these modalities enhances valence regression performance, while the impact on arousal regression is less significant. These findings not only corroborate previous research in other languages but also enrich the understanding of emotional expression in Finnish speech.
Combining text and audio features reveals a significant boost in modeling emotional valence, but not arousal, in Finnish speech.
Existing research on affect in speech has shown how acoustic surface characteristics and content-related linguistic aspects of speech both relate to perceived emotional arousal and valence. However, it is not clear what the relative contributions of these two factors are in the perceptual process. This is especially true for Finnish, for which most existing studies focus on either acoustic-phonetic or text analysis. This paper presents a study where we systematically explore the combinatory role of text- and audio-based features in modeling the human perception of valence and arousal using a newly released affective speech corpus for spontaneous Finnish. We show that the combination of text- and audio-based features improves valence regression results over the individual modalities, whereas for arousal regression the complementary effect is not substantial. The results support prior findings from other languages, providing new data and knowledge on spontaneous Finnish speech.