Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper investigates how humans and LLMs differ in their interpretation of plausible and implausible subject-verb-object events, using a dataset of event triples with varying levels of abstractness and concreteness. The study finds that humans are adept at nuanced detection and contextualization of both non-literal and implausible events, while LLMs tend to favor plausible, non-literal interpretations even when events are highly implausible. This suggests LLMs lack a robust understanding of real-world plausibility constraints.
LLMs will readily twist implausible events into plausible metaphors, revealing a surprising gap in their grasp of real-world constraints compared to humans.
This work explores the connection between (non-)literalness and plausibility at the example of subject-verb-object events in English. We design a systematic setup of plausible and implausible event triples in combination with abstract and concrete constituent categories. Our analysis of human and LLM-generated judgments and example contexts reveals substantial differences between assessments of plausibility. While humans excel at nuanced detection and contextualization of (non-)literal vs. implausible events, LLM results reveal only shallow contextualization patterns with a bias to trade implausibility for non-literal, plausible interpretations.