Search papers, labs, and topics across Lattice.
This paper investigates the impact of targeted user training on the adoption and productive use of generative AI in legal analysis, using a randomized study with 164 law students completing an issue-spotting examination. The study compared no GenAI access, optional LLM access, and optional LLM access with a brief training intervention, finding that training significantly increased LLM adoption and improved examination performance by 0.27 grade points. Principal stratification suggests that training primarily expands the scope of GenAI use rather than enhancing effectiveness among existing users.
A mere 10-minute training session can boost GenAI adoption rates by 58% and improve performance by a third of a letter grade in legal analysis tasks.
Can targeted user training unlock the productive potential of generative artificial intelligence (GenAI) in professional settings? We investigate this question using a randomized study involving 164 law students completing an issue-spotting examination. Participants were assigned to one of three conditions: no GenAI access, optional access to a large language model (LLM), or optional access accompanied by an approximately ten-minute training intervention. Training significantly increased LLM adoption--the usage rate rose from 26% to 41%--and improved examination performance. Students with trained access scored 0.27 grade points higher than those with untrained access (p = 0.027), equivalent to roughly one-third of a letter grade. By contrast, access to an LLM without training did not improve performance and was associated with shorter answers relative to no access. Using principal stratification, we decompose the overall effect into adoption and effectiveness channels. Point estimates are consistent with training operating primarily by expanding the scope of GenAI use rather than by enhancing effectiveness among existing users, though confidence intervals are wide. Overall, our findings provide evidence that complementary investments in user training are critical for realizing GenAI productivity gains in knowledge-intensive fields where concerns about reliability may inhibit adoption.
