Search papers, labs, and topics across Lattice.
This study investigates the impact of AI assistance on human skill development through a controlled experiment with logic puzzles, revealing a complex relationship between short-term performance boosts and long-term skill acquisition. By manipulating the cost of AI assistance, the researchers found that lower costs led to increased reliance on AI, which subsequently resulted in poorer performance once the assistance was removed. The findings, supported by a Bayesian latent ability model, suggest that independent reasoning during AI-access periods is crucial for skill development, as excessive reliance on AI can hinder the growth of latent abilities.
Over-reliance on AI assistance can significantly undermine long-term skill development, leading to a false sense of competence in problem-solving tasks.
While AI assistance can improve human task performance in the short term, it may also undermine the development of skills in the longer term. We examine this tension in a controlled logic-puzzle experiment involving on-demand AI assistance, where participants complete tasks before, during, and after AI is available. By experimentally varying AI request costs, we find that lower-cost assistance induces more frequent AI use. We also find that participants who request AI assistance during the AI-access phase perform worse at the task after assistance is removed, and their subsequent unassisted performance is overestimated when predicted from earlier AI-assisted performance. We use a Bayesian latent ability model to separate initial ability, post-AI ability, and participant-specific skill change, while estimating how independent reasoning during the AI-access phase relates to skill development. The results show that greater independent problem-solving effort is associated with larger gains in latent ability, consistent with the interpretation that skill development is weaker when AI assistance substitutes for independent reasoning.
