Search papers, labs, and topics across Lattice.
This paper investigates the detectability of AI-generated persuasive text compared to human-written persuasion, motivated by concerns about LLM misuse. The authors introduce Persuaficial, a multilingual benchmark for persuasive text in six languages, and evaluate the performance of automatic detection models. Results show that while overt AI-generated persuasion is easier to detect, subtle AI-generated persuasion degrades detection performance, highlighting the challenges in distinguishing AI and human-generated persuasive content.
Subtle AI-generated persuasion can slip past current detection methods, making it harder to spot than human-written manipulation.
Large Language Models (LLMs) can generate highly persuasive text, raising concerns about their misuse for propaganda, manipulation, and other harmful purposes. This leads us to our central question: Is LLM-generated persuasion more difficult to automatically detect than human-written persuasion? To address this, we categorize controllable generation approaches for producing persuasive content with LLMs and introduce Persuaficial, a high-quality multilingual benchmark covering six languages: English, German, Polish, Italian, French and Russian. Using this benchmark, we conduct extensive empirical evaluations comparing human-authored and LLM-generated persuasive texts. We find that although overtly persuasive LLM-generated texts can be easier to detect than human-written ones, subtle LLM-generated persuasion consistently degrades automatic detection performance. Beyond detection performance, we provide the first comprehensive linguistic analysis contrasting human and LLM-generated persuasive texts, offering insights that may guide the development of more interpretable and robust detection tools.
