Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This study developed seven Retrieval-Augmented Generation (RAG) models informed by leading deception theories and evaluated their performance against baseline models in detecting deception across 700 statements from five datasets. The findings revealed that while RAG models achieved a slightly higher accuracy (57.0%) compared to baseline models (54.6%), both types of models exhibited similar overall detection accuracies comparable to human performance. Interestingly, the theoretical framework influenced response bias significantly, with some theories leading to a strong bias towards truth or lies, suggesting that while current RAG models are unreliable for deception detection, they hold potential for improvement with further refinement.
RAG models show a slight edge in deception detection but reveal troubling biases that challenge their reliability in real-world applications.
The current work developed seven Retrieval-Augmented Generation (RAG) models based on leading deception theories and compared how deception judgments were made relative to baseline models. Across 700 statements drawn from five published deception datasets, four large language models (gpt-4o, claude-sonnet-4-6, ollama/llama3, deepseek-v4-flash), and two run-types (RAG vs. baseline), a total of 39,200 deception judgments were rendered. Detection accuracies were consistent with typical human accuracies and not statistically different across RAG (54.5%) and baseline models (54.6%). RAG-based models (57.0%) were less truth-biased than baseline models (59.7%), but the effect size was quite small. Theoretical perspective mattered little for accuracy yet mattered substantially for response bias, which ranged from highly lie-biased (the verifiability approach, 32.2%) to highly truth-biased (truth-default theory, 88.1%). Content effects and model effects further moderated the results. Theory-guided AI judgments are unreliable with current parameters, yet they might show promise with additional datasets, model testing, and theory-to-data matching.