Search papers, labs, and topics across Lattice.
This paper investigates the factors influencing user reliance on conversational AI for information search, even when web search is available for verification. Through a mixed-subjects experiment with warm and neutral chatbots, the study finds that reliance persists, driven primarily by pre-existing user perceptions rather than answer accuracy. Notably, a warm conversational style indirectly increases reliance by boosting agreement with incorrect chatbot responses.
Even when users can easily fact-check, their pre-existing beliefs about chatbots, not the chatbot's accuracy, dictate whether they blindly trust AI answers, and a "warm" chatbot voice only makes the problem worse.
Conversational artificial intelligence (AI) provides an efficient and convenient gateway to information access. However, it can cause overreliance when users blindly trust AI and accept its answers without fact-checking. Information search increasingly follows a hybrid interaction paradigm that combines conversational AI with web search, making fact-checking easier. In this paper, we examine whether this interaction paradigm is effective in curbing reliance. We further investigate the underlying factors (e.g., digital literacy and conversation warmth) that drive users to verify AI answers. We conduct a mixed-subjects question-answering experiment where participants interact with either a warm or a neutral chatbot. Our findings reveal that reliance persists despite users having access to both conversational and web search. The decision to verify is driven primarily by existing user perceptions (e.g., prior trust in chatbots) rather than answer properties, with some users fact-checking regardless of the context and others trusting chatbots by default. Warm conversational style has an indirect yet critical influence on reliance by increasing agreement with the chatbot when it is incorrect. Consulting additional AI sources predicts higher accuracy, while traditional web search does not. Our study extends overreliance research by: (a) demonstrating its persistence despite access to fact-checking, (b) identifying verification behavior as user-dependent, and (c) revealing conversational warmth's indirect effect on overreliance with implications for designing trustworthy conversational search systems.
