Search papers, labs, and topics across Lattice.
This study investigates the impact of generative AI tools, specifically ChatGPT, on informal learning and information-seeking behaviors compared to traditional search engines like Google. Through a field experiment involving daily diary entries over eight days, it was found that participants using ChatGPT experienced reduced agency and greater meta-cognitive load due to reliance on AI for information selection, leading to poorer learning outcomes, particularly in critical thinking. The research highlights significant biases in ChatGPT outputs and a shift in user behavior that limits exploration of knowledge, raising concerns about the implications of AI on cognitive processes and learning efficacy.
Relying on ChatGPT for information seeking may diminish users' agency and critical learning outcomes, revealing hidden risks of generative AI in education.
Generative AI (GenAI) tools offer increasing opportunities for augmenting human cognitive tasks. Among these tasks, information seeking is being rapidly reshaped by GenAI tools, with potentially profound implications for learning and knowledge acquisition. To investigate these implications, we conducted a between-subjects field experiment in which participants pursued informal learning by seeking information through either ChatGPT or Google Search over a span of 8 days. Using a daily diary protocol, we gathered in-situ data on their information-seeking processes. Our findings show that participants in the ChatGPT group experienced diminished agency in their information-seeking processes, as they offloaded much of the information selection to AI, and consequently experienced greater meta-cognitive load arising from this reduced sense of control. We further highlight two sources of distortion in information access when using ChatGPT: biases in ChatGPT outputs, particularly towards providing solution-oriented artifacts over principled knowledge; and systematic shifts in users'information-seeking behaviors, whereby the conversational and socially-oriented interaction paradigm of current GenAI tools may inadvertently reduce exploration of the broader knowledge space. As a result, on average, participants in the ChatGPT group had worse learning outcomes than those using Google, especially for higher-order critical learning. Our work suggests inherent tensions between offloading information seeking to AI and meaningful learning, and provides broader implications for understanding AI's risks to human cognition.
Microsoft Research