Search papers, labs, and topics across Lattice.
This study evaluated a 45-minute intervention designed to teach prompt-based programming to engineering students, comparing it to a traditional code tracing activity. While the experimental group showed a modest improvement in their ability to specify computational goals to large language models (LLMs), the increase in prompting self-efficacy was significantly greater than the control group. These findings indicate that even brief instructional interventions can enhance learners' interactions with LLMs, though more extensive practice may be necessary for substantial skill development.
A quick 45-minute lesson can significantly boost students' confidence in using LLMs, but the actual improvement in skill remains modest.
Prompt-based programming, a new modality enabled by large language models (LLMs), allows users to express computational goals through natural language rather than traditional code. While this approach lowers barriers to entry, especially for non-CS learners, it does not eliminate the need for foundational CS skills. Learners often struggle to communicate their intent clearly to LLMs, resulting in vague or underspecified prompts. Prior work has documented the need for explicit prompting for both CS and non-CS learners. However, it remains less clear how such instruction can fit into busy classrooms or how much time is needed to produce meaningful gains. In this paper, we evaluated a 45-minute prompt-based programming intervention, consisting of a lesson with guided practice, against a business-as-usual CS lab activity (code tracing) of equal length, representing a class without prompt-focused instruction. We conducted a randomized controlled study with 55 engineering students. We found that students in the experimental condition improved more on average (though not significantly more) from pre- to post-test than the control group (+10.8 vs +1.1 percentage points) and showed significantly greater average gains in prompting self-efficacy (+35.4 vs +21.9 percentage points). Our results suggest it is likely that a brief intervention can improve learners'ability to specify computational goals to LLMs. However, the effect was modest, suggesting that prompting skills may require more time and practice to develop. We provide a lightweight lesson that requires no prior CS background and can be readily dropped into existing courses.
