Search papers, labs, and topics across Lattice.
University of Padua, Italy, Polish-Japanese Academy of Information Technology, Poland, NASK National Research Institute, Poland, NASK National Research Institute, Poland Correspondence: contact@amodzelewski.com Abstract Large Language Models (LLMs) can generate highly persuasive text, raising concerns about their misuse for propaganda, manipulation, and other harmful purposes. This leads us to our central question: Is LLM-generated persuasion more difficult to automatically detect than human-written persuasion? To address this, we categorize controllable generation approaches for producing persuasive content with LLMs and introduce Persuaficial, a high-quality multilingual benchmark covering six languages: English, German, Polish, Italian, French and Russian. Using this benchmark, we conduct extensive empirical evaluations comparing human-authored and LLM-generated persuasive texts. We find that although overtly persuasive LLM-generated texts can be easier to detect than human-written ones, subtle LLM-generated persuasion consistently degrades automatic detection performance. Beyond detection performance, we provide the first comprehensive linguistic analysis contrasting human and LLM-generated persuasive texts, offering insights that may guide the development of more interpretable and robust detection tools. Can AI-Generated Persuasion Be Detected? Persuaficial Benchmark and AI vs. Human Linguistic Differences Arkadiusz Modzelewski1,2,3, Pawe艂 Golik, Anna Ko艂os3, Giovanni Da San Martino1
1
0
3
Knowing the *intent* behind disinformation can significantly improve LLMs' ability to detect it, paving the way for more robust defenses against malicious narratives.
