Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces a novel method for generating multiple-choice questions (MCQs) from input documents by constructing a knowledge graph (KG) using an LLM, then systematically generating questions by sampling triples/quintuples from the KG and prompting another LLM to create the stem. The approach incorporates nine difficulty signals derived from the KG structure and LLM outputs, which are combined into a unified difficulty score. Experiments show the generated MCQs are high-quality and the estimated difficulty aligns with human judgment, improving automated MCQ generation.
Forget hand-crafted features: this system learns to predict MCQ difficulty directly from knowledge graph structure and LLM outputs, matching human perception.
Generating multiple-choice questions (MCQs) with difficulty estimation remains challenging in automated MCQ-generation systems used in adaptive, AI-assisted education. This study proposes a novel methodology for generating MCQs with difficulty estimation from the input documents by utilizing knowledge graphs (KGs) and large language models (LLMs). Our approach uses an LLM to construct a KG from input documents, from which MCQs are then systematically generated. Each MCQ is generated by selecting a node from the KG as the key, sampling a related triple or quintuple -- optionally augmented with an extra triple -- and prompting an LLM to generate a corresponding stem from these graph components. Distractors are then selected from the KG. For each MCQ, nine difficulty signals are computed and combined into a unified difficulty score using a data-driven approach. Experimental results demonstrate that our method generates high-quality MCQs whose difficulty estimation is interpretable and aligns with human perceptions. Our approach improves automated MCQ generation by integrating structured knowledge representations with LLMs and a data-driven difficulty estimation model.