Search papers, labs, and topics across Lattice.
This paper investigates whether Large Language Models (LLMs) trained with human feedback are susceptible to motivated reasoning, specifically suppressing fraud warnings when pressured by investors already convinced of an opportunity. Through a pre-registered experiment involving seven LLMs and twelve investment scenarios, the study compared AI and human advisors' responses to legitimate, high-risk, and fraudulent opportunities. The key finding is that LLMs consistently outperformed humans in fraud detection, showing no suppression of warnings under pressure and exhibiting a 0% endorsement rate for fraudulent investments, compared to 13-14% for humans.
LLMs are surprisingly immune to motivated reasoning in investment advice, flagging fraud that human advisors miss even when facing pressure from biased investors.
Large language models trained on human feedback may suppress fraud warnings when investors arrive already persuaded of a fraudulent opportunity. We tested this in a preregistered experiment across seven leading LLMs and twelve investment scenarios covering legitimate, high-risk, and objectively fraudulent opportunities, combining 3,360 AI advisory conversations with a 1,201-participant human benchmark. Contrary to predictions, motivated investor framing did not suppress AI fraud warnings; if anything, it marginally increased them. Endorsement reversal occurred in fewer than 3 in 1,000 observations. Human advisors endorsed fraudulent investments at baseline rates of 13-14%, versus 0% across all LLMs, and suppressed warnings under pressure at two to four times the AI rate. AI systems currently provide more consistent fraud warnings than lay humans in an identical advisory role.
