Search papers, labs, and topics across Lattice.
This study evaluates the capability of large language models (LLMs) to interpret negation within figurative language by creating new annotations for an existing dataset. The findings reveal that the interplay of negation and figurative expressions poses significant challenges for LLMs, with performance varying notably based on the style of prompts employed. This research highlights the nuanced difficulties LLMs face in understanding complex linguistic constructs, which is crucial for improving their effectiveness in real-world applications.
LLMs struggle significantly with interpreting negation in figurative language, revealing prompt style as a critical factor in their performance.
Figurative language and negation are two areas that challenge current language models, however, both are widely used throughout written and spoken language. Large language models (LLMs) are also widely used in everyday contexts where they cannot necessarily be tuned for a specific dataset. It is therefore essential to understand the ability of LLMs to correctly interpret text that includes both negation and figurative language. To investigate this, we develop a set of new annotations to an existing dataset of figurative language, and test a range of language models on the dataset. We find that the combination of negation and figurativeness can present a particular challenge, and that performance overall and across different negation types is particularly dependent on the prompt style used.
