Search papers, labs, and topics across Lattice.
This study introduces Clover, a novel AI code completion tool that logs student interactions and incorporates attention checks to assess critical engagement during programming tasks. By analyzing interaction patterns, the authors found that a higher frequency of accepting code suggestions via tab completion correlated with poorer performance on attention checks, while increased dwell time on suggestions indicated better engagement. These findings highlight the importance of reflective engagement in AI-assisted programming and suggest that monitoring behavioral signals can enhance learning outcomes.
Students who rapidly accept AI code suggestions are less likely to engage critically, revealing a potential pitfall in AI-assisted learning environments.
AI code completion tools, such as Github Copilot, provide students with code suggestions to help them write programs. However, recent qualitative studies suggest that students fail to critically evaluate these suggestions. We present Clover, a code completion tool that logs students'interactions with code suggestions and additionally offers attention checks to probe reflective engagement during programming tasks. We also develop a taxonomy of behavioral interaction metrics for AI-assisted programming, informed by literature. We analyzed relationships between interaction patterns, engagement with attention checks, and task performance. We observed that higher rates of tab accept were associated with lower attention check performance, while increased dwell time was associated with higher attention check performance. We conclude by discussing how programming process data and attention checks might support reflective engagement in AI-assisted programming.
