Search papers, labs, and topics across Lattice.
This study critiques traditional word-level grievance lexicons, which fail to account for the nuanced context of terms, leading to misleading evaluations of grievance in online texts. By employing contextual models that read surrounding text, the authors demonstrate significant improvements in grievance detection, particularly in cases where lexicons are silent. The findings reveal that context-aware approaches yield a more accurate representation of grievances across five languages, enhancing average precision from 0.14 to 0.20 on lexicon-negative instances.
Contextual models outperform traditional grievance lexicons, revealing that reading surrounding text can significantly enhance detection accuracy, especially in nuanced cases.
Grievance is one of the warning signs analysts look for when assessing threats of violence. It is increasingly measured at scale from online text, most often with word-level lexicons like the Grievance Dictionary that score by matching weighted terms. Such matching is a fast and transparent proxy, but it cannot resolve whether a term is asserted, quoted, negated, or condemned. These lexicons are also often evaluated on pools enriched with the very examples they retrieve, so a high score partly reflects agreement with the lexicon's own selection rule. Examining a five-language, 2{,}000-item evaluation pool, we find its halves separated almost perfectly by the lexicon itself: every item labeled ``random'' is in fact lexicon-negative, so the lexicon's apparent macro-AUROC of 0.686 collapses to a 0.500 floor fixed by construction. We keep the dictionary's 22-construct ontology but replace term matching with context-reading models, evaluated on a non-circular benchmark that separates unconditional-random, lexicon-positive, and lexicon-negative strata across five languages. Reading the full post rather than the target sentence alone helps most where the lexicon is silent, raising average precision on lexicon-negative text from 0.14 to 0.20, with the largest gains on quoted, implicit, and cross-sentence grievance. Together, these results show that grievance is measured more faithfully by reading the surrounding context, and more honestly when tested on text the lexicon did not select. We release our code and benchmark at https://github.com/behavioral-ds/multilingual_grievance.
