Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces a novel resource of 1,482 tweets annotated for enthymemes in politically controversial discourse, addressing the subjective nature of argument annotation. By establishing clear guidelines based on Walton's argumentation schemes, the authors allow for interpretative flexibility while revealing the benefits of label variation in model training. Notably, models trained on diverse annotator perspectives outperform those relying on majority-vote labels, highlighting the importance of embracing disagreement in NLP tasks.
Models leveraging diverse annotator perspectives can significantly outperform those trained on consensus labels, revealing the hidden value of disagreement in enthymeme detection.
Enthymemes, arguments with unstated premises or conclusions, are pervasive in persuasive discourse, yet their annotation remains notoriously subjective. We present a resource of 1,482 tweets from politically controversial discourse, annotated by five annotators for the presence of enthymemes and their argument structure, designed to study label variation. We first revisit the definition of enthymemes and propose annotation guidelines anchored in Walton's argumentation schemes, offering a structured and constrained approach that nonetheless preserves room for the interpretive nature of the task. This contrasts with past resources, which tend to eliminate disagreement, obscuring its sources and preventing investigation of its potential benefits for model performance. We further propose a complexity analysis of the task, identifying where annotation imposes high cognitive load and may give rise to inconsistent annotation. Our preliminary experiments show that models trained on annotator disagreement outperform models trained on hard majority-vote labels. We close by reflecting on how structural openness in enthymeme definitions and guidelines enables the study of variation in subjective inferential processes for future resources and downstream NLP applications concerned with human inference.