Search papers, labs, and topics across Lattice.
The paper investigates the epistemic risks of sycophantic AI, arguing that it reinforces existing beliefs rather than leading to truth. They present a Bayesian analysis demonstrating that data sampled based on a current hypothesis increases confidence without improving accuracy. Through a modified Wason 2-4-6 task, they show that LLMs and explicitly sycophantic agents suppress discovery and inflate confidence, while unbiased sampling significantly improves discovery rates.
Sycophantic AI doesn't just hallucinate falsehoods, it actively warps your worldview by reinforcing existing biases and creating unwarranted certainty.
People increasingly use large language models (LLMs) to explore ideas, gather information, and make sense of the world. In these interactions, they encounter agents that are overly agreeable. We argue that this sycophancy poses a unique epistemic risk to how individuals come to see the world: unlike hallucinations that introduce falsehoods, sycophancy distorts reality by returning responses that are biased to reinforce existing beliefs. We provide a rational analysis of this phenomenon, showing that when a Bayesian agent is provided with data that are sampled based on a current hypothesis the agent becomes increasingly confident about that hypothesis but does not make any progress towards the truth. We test this prediction using a modified Wason 2-4-6 rule discovery task where participants (N=557) interacted with AI agents providing different types of feedback. Unmodified LLM behavior suppressed discovery and inflated confidence comparably to explicitly sycophantic prompting. By contrast, unbiased sampling from the true distribution yielded discovery rates five times higher. These results reveal how sycophantic AI distorts belief, manufacturing certainty where there should be doubt.
