Search papers, labs, and topics across Lattice.
This paper addresses the limitations of traditional moral classification models in NLP, which often aggregate multiple annotators' labels into a single ground truth, thereby neglecting the subjective nature of moral judgments. By extending a pretrained language model with a layer that captures annotator-specific features, the authors improve the accuracy of individual annotation predictions and uncover valuable insights into the moral perspectives of annotators. The findings reveal that conventional models can obscure the inherent variability in moral viewpoints, suggesting that recognizing individual differences enhances the understanding of moral classification tasks.
Aggregating moral judgments can obscure critical individual perspectives, but modeling these differences reveals deeper insights into moral classification.
Understanding moral values in social media text offers insight into moral judgement formation, and supervised NLP models trained on crowdsourced data have achieved strong classification performance. However, most approaches simplify the problem by aggregating multiple annotators' labels into a single "ground truth", overlooking the inherent subjectivity of the task. In practice, there are disagreements between annotators caused by personal viewpoint or inherent ambiguities, particularly for short tweets. Here, we extend a pretrained language model with a layer that learns annotator-specific features. Our model improves predictions of individual annotations and yields representations that reveal meaningful insights into annotators' moral perspectives. We show that models trained on aggregated labels may hide variation and give a misleading impression of performance. Overall, we demonstrate that disagreement reflects the inherent subjectivity of the task and that modelling individual perspectives creates benefits for moral classification of texts.
