Search papers, labs, and topics across Lattice.
This survey analyzes the integration of Large Language Models (LLMs) across the four stages of the Abuse Detection Lifecycle (ADL): Label & Feature Generation, Detection, Review & Appeals, and Auditing & Governance. It synthesizes research and industry practices, highlighting architectural considerations for production deployment and examining the strengths and limitations of LLM-driven approaches in each stage. The survey identifies key challenges such as latency, cost-efficiency, and adversarial robustness, outlining future research directions for operationalizing LLMs in large-scale abuse detection.
LLMs are poised to revolutionize online abuse detection, but realizing their potential requires tackling critical challenges like latency, cost, and adversarial attacks across the entire abuse detection lifecycle.
Online abuse has grown increasingly complex, spanning toxic language, harassment, manipulation, and fraudulent behavior. Traditional machine-learning approaches dependent on static classifiers and labor-intensive labeling struggle to keep pace with evolving threat patterns and nuanced policy requirements. Large Language Models introduce new capabilities for contextual reasoning, policy interpretation, explanation generation, and cross-modal understanding, enabling them to support multiple stages of modern safety systems. This survey provides a lifecycle-oriented analysis of how LLMs are being integrated into the Abuse Detection Lifecycle (ADL), which we define across four stages: (I) Label \&Feature Generation, (II) Detection, (III) Review \&Appeals, and (IV) Auditing \&Governance. For each stage, we synthesize emerging research and industry practices, highlight architectural considerations for production deployment, and examine the strengths and limitations of LLM-driven approaches. We conclude by outlining key challenges including latency, cost-efficiency, determinism, adversarial robustness, and fairness and discuss future research directions needed to operationalize LLMs as reliable, accountable components of large-scale abuse-detection and governance systems.
