Search papers, labs, and topics across Lattice.
This paper advocates for the use of structural causal modeling in educational assessments to enhance the evaluation of student competencies and the design of targeted interventions. By shifting from traditional psychometric models to a framework that supports interventional and counterfactual reasoning, the authors demonstrate how this approach can provide insights into the effectiveness of specific educational strategies, such as the use of hints. The proposed protocol, illustrated with data from assessments of algorithmic skills, shows that logical elicitation of structural equations can yield more robust and actionable insights than standard probabilistic models.
Shifting to structural causal modeling reveals how targeted interventions can be explicitly evaluated, unlocking new insights into student learning dynamics.
Accurate assessment of student competencies is essential for enabling educators to identify individual needs, design targeted interventions, and evaluate the effectiveness of educational strategies. Empirical assessment procedures are typically grounded in psychometric models, such as item response theory, which relate student competence levels to performance on assessment tasks. In this paper, we advocate adopting a structural causal modelling approach to educational assessment, moving beyond probabilistic belief updating toward a framework that explicitly supports interventional and counterfactual reasoning. We propose a corresponding protocol for its construction and analyse the practical relevance of forms of reasoning that remain inaccessible to standard associative models, including the explicit modelling of interventions such as hints and the related counterfactual scenario analysis. Although our protocol requires the structural equations to be elicited from experts, the necessary information is purely logical and does not rely on probabilistic, less tenable assumptions. We illustrate the approach using data from an assessment that employs complex tasks designed to measure compulsory school student algorithmic skills.
