Search papers, labs, and topics across Lattice.
This study explores how various forms of conceptual grounding influence the detection and reasoning of antisemitism in four state-of-the-art LLMs. By employing expert-annotated datasets, the authors reveal that fine-grained taxonomic representations significantly enhance recall but compromise precision, while larger conceptual resources do not yield additional benefits. The research identifies persistent challenges in detecting post-Holocaust antisemitism and highlights systematic limitations in LLM explanations, such as overconfidence and reliance on lexical cues.
Fine-grained taxonomic representations can boost recall in antisemitism detection, but at the cost of precision, revealing a critical trade-off in LLM performance.
LLMs enable the integration of external conceptual resources at inference time, creating new opportunities for detecting ideologically and historically complex phenomena such as antisemitism. We investigate how different forms of conceptual grounding affect antisemitism detection and explanation behavior across four state-of-the-art LLMs. Using two expert-annotated datasets, we compare definitional, fine-grained taxonomic, example-augmented, and large-context representations of antisemitism. We find that fine-grained taxonomic representations substantially improve recall, while simultaneously reducing precision. Surprisingly, supplying substantially larger conceptual resources yields no additional quantitative benefit. Post-Holocaust antisemitism poses the most persistent challenge across models and configurations. Analysis of explanations further reveals systematic limitations including overproduction of conceptual references, reliance on lexical cues, overconfidence, and difficulties with subtle or justificatory forms of antisemitism. Our findings highlight both the potential and the remaining limitations of conceptually grounded LLMs for antisemitism detection and reasoning.
