Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This pilot study evaluates the effectiveness of AI-generated interactive fiction (IF) in higher education by assessing user perceptions of narrative clarity, coherence, and engagement. Participants rated the generated episodes positively for narrative clarity and length acceptance, but found story-content coherence lacking, highlighting specific areas for improvement such as quiz integration and narrative consequences. The findings suggest that while AI can create engaging educational content, significant refinements are necessary to enhance coherence and overall user experience.
Users found AI-generated interactive fiction engaging but flagged serious coherence issues, revealing critical design flaws for educational applications.
Generative artificial intelligence (AI) can produce educational content at scale, including interactive and narrative learning experiences, but technical generation alone is not sufficient: scenarios that are confusing, narratively inconsistent, or unengaging are unlikely to be useful in practice. This paper presents a pilot user-centred evaluation of AI-generated interactive fiction (IF) for educational use in higher education. Using a previously described domain-agnostic pipeline and a shared STEM content base, we generated a controlled pool of scenarios and asked participants (N = 22, STEM higher-education) to play one generated episode and rate it on narrative clarity, story-content coherence, engagement, and length acceptance. A free-text prompt captured open feedback. Narrative clarity and length acceptance were rated positively, engagement sat near the neutral mid-point of the scale, and story-content coherence was the weakest dimension by a clear margin. Qualitative feedback points to quiz integration as the bottleneck. Artificial in-fiction motivation for quiz prompts and abrupt setting changes were reported. Feedback also pointed to missing story-level consequences for wrong answers. From these observations, we derive concrete design implications that can inform larger follow-up studies, including later work on learning effectiveness.