Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper addresses the rising threat of conversational scams by developing an explainable agentic system that detects these scams over extended interactions rather than isolated messages. The system achieves 100% recall on single-message phishing detection and identifies all conversational scams in the LoveFraud02 corpus, achieving 97.8% accuracy on the newly introduced ConScamBench-278 benchmark. User studies indicate that the system significantly enhances users' trust and confidence in identifying suspicious conversations, highlighting its practical utility in real-world applications.
Conversational scams can be detected with 97.8% accuracy by leveraging context across multiple interactions, a significant leap beyond traditional single-message detection methods.
Following the rapid progress of generative Artificial Intelligence, there is a growing threat posed by conversational scams. These scams often span over multiple weeks or months, gradually build trust and request for money or sensitive information. Existing scam-detection systems mainly focus on isolated messages, which renders them inadequate against this evolving threat. This paper extends single-message phishing detection and presents an explainable agentic system for detecting sophisticated conversational scams. It also introduces ConScamBench-278, an initial public multi-category benchmark for conversational scam detection spanning eight scam types, released to support reproducible evaluation and future expansion. On isolated messages the single-message detector attains 100% phishing recall, while the conversation-level detector identifies all conversational scams in the public LoveFraud02 corpus (83/83) and reaches 97.8% accuracy (95% CI [95.4, 99.0]) on ConScamBench-278. Two user studies (N = 100 and N = 45) further motivate the system: participants report frequently experiencing uncertainty when judging suspicious conversations. In an uncontrolled pre/post comparison, users self-reported trust, self-confidence, and perceived need for AI-based scam detection all increased (p<0.001, Wilcoxon signed-rank). The system also receives a System Usability Scale score of 74.7 (95% CI [72.5, 76.9]), above the established usability benchmark.