Search papers, labs, and topics across Lattice.
This paper benchmarks the ability of three SLMs (EuroLLM, Aya Expanse, and Gemma) to preserve fine-grained emotions during backtranslation across five European languages using the GoEmotions dataset. They find that emotion-aware prompting improves emotional preservation, and that ModernBERT performs comparably to BERT for emotion classification in this MT evaluation task. The study highlights the challenges SLMs face in maintaining affective nuance, even with targeted prompting strategies.
Even with emotion-aware prompting, today's best small language models still struggle to preserve subtle emotional nuances when translating between languages.
Preserving affective nuance remains a challenge in Machine Translation (MT), where semantic equivalence often takes precedence over emotional fidelity. This paper evaluates the performance of three state-of-the-art Small Language Models (SLMs) -- EuroLLM, Aya Expanse, and Gemma -- in maintaining fine-grained emotions during backtranslation. Using the GoEmotions dataset, which comprises Reddit comments across 28 distinct categories, we assess emotional preservation across five European languages: German, French, Spanish, Italian, and Polish. Specifically, we investigate (i) the inherent capability of these SLMs to retain emotional sentiment, (ii) the efficacy of emotion-aware prompting in improving preservation, and (iii) the performance of ModernBERT as a contemporary alternative to BERT for emotion classification in MT evaluation.
