Search papers, labs, and topics across Lattice.
This study evaluates two methods for assessing student teams during tabletop exercises (TTXs) in computing education, focusing on clustering and large language models (LLMs). By analyzing data from 81 participants, the researchers found that clustering provided timely, targeted feedback based on team performance, while LLMs, particularly GPT-5.2, showed improved accuracy in assessing communication compared to instructor scores. The integration of these methods into the INJECT platform enhances scalability and effectiveness in teaching practices for crisis response training.
Clustering methods can streamline feedback for student teams, while advanced LLMs like GPT-5.2 significantly improve assessment accuracy in computing education.
This full paper in the research-to-practice track presents methods for assessing student teams in tabletop exercises (TTXs). TTXs enable learner teams to prepare for workplace tasks and practice crisis responses, such as resolving cybersecurity incidents. While assessment is essential for determining how well teams achieve learning objectives, the complex, open-ended nature of TTXs often leads to delayed or incomplete feedback. TTX learning platforms can record teams'actions and communication; yet, leveraging these data to assess performance is underexplored. To address this gap, we compared two post-TTX team assessment methods -- clustering and large language models (LLMs) -- using an original dataset from 81 participants across two countries. We evaluated these methods against instructor-assigned scores based on standardized rubrics. Clustering grouped teams that approached TTX tasks similarly, enabling instructors to deliver faster, targeted feedback to teams within a cluster. This method was valid and reliable, with low computational requirements. LLMs used the standardized rubrics to assess teams'communication. While GPT-4o frequently disagreed with instructor scores, GPT-5.2 demonstrated considerably lower error. The researched methods have been integrated into INJECT, an open-source TTX learning platform, to support scalability and teaching practice. To encourage community adoption, we publicly share all datasets, software tools, and a full-fledged TTX scenario.
