Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This study investigates human-AI collaboration in a competitive question-answering game, focusing on two reliance decisions: delegation and adoption. By analyzing 24 matches between expert humans and AI agents, the research reveals that while collaboration outperforms individual efforts, humans frequently make suboptimal decisions—under-relying on correct AI suggestions and over-relying on misleading ones. Key findings indicate that confirmation bias and misalignment in confidence levels contribute to these reliance issues, suggesting that improved trust mechanisms are essential for effective human-AI teamwork.
Humans miss 3.9% of opportunities to leverage correct AI suggestions while also over-relying on misleading outputs, highlighting critical gaps in trust and decision-making in human-AI collaboration.
AI systems are fallible, and humans can make mistakes in deciding whether to trust AI over their own judgment. Thus, improving human-AI collaboration requires understanding when, why, and how humans decide to rely on AI. We study two distinct reliance decisions: the delegation choice -- deciding when to let AI act autonomously without knowing its output, and the adoption choice -- evaluating AI suggestions and deciding how to use them. Both of these decoupled reliance patterns shape collaboration, but prior work rarely studies them together in realistic settings with the same users. We address this gap by studying collaborative human--AI teams competing in a question-answering game in which humans can choose when and how to work with AI agents to win. Our 24 matches pair 23 expert humans with 16 AI agents, capturing 387 delegation and 1440 adoption decisions. While human--AI collaboration performs better than either AI or humans alone, humans make suboptimal collaboration decisions, both under-relying on correct AI suggestions (3.9% of opportunities missed) and over-relying when AI misleads them (1.7%). Both parties contribute wrong answers: reported model confidence is near chance when humans and AI disagree, while confirmation bias drives higher under-reliance (64.5%) when an AI suggestion agrees with humans'initial incorrect answer. To close this gap, we recommend calibrated confidence, evidence-grounded explanations, and mechanisms that help users refine trust.
Google Research