Search papers, labs, and topics across Lattice.
This paper replicates and extends a 2019 systematic literature review (SLR) and a survey to assess the state of replication studies in Computing Education Research (CER). The SLR identifies 63 replication studies out of 2,516 papers published between 2019 and 2025, indicating a marginal increase in replication efforts compared to the previous decade. The survey reveals that while researchers engage in replication internally, publishing replication studies remains challenging, hindering the growth of verifiable knowledge in CER.
Despite increased discussion around open science, replication studies in computing education research have only seen marginal growth, suggesting a disconnect between espoused values and actual research practices.
Replication studies play an important role in Computing Education Research (CER) by supporting the development of consistent and reliable scientific knowledge. However, prior research indicates that the CER community tends to prioritise novel contributions over replication. A 2019 SLR identified only 54 replication studies among 2,269 papers published between 2009 and 2018 across five major CER venues. This study aims to examine how the landscape of replication research in CER has evolved since 2019. Specifically, we investigate whether the prevalence of replication studies has increased and explore current perceptions and experiences of CER researchers regarding replication. We replicated two prior studies. First, we conducted an updated SLR to identify replication studies published between 2019 and 2025 in the same five CER venues. Second, we replicated a survey of Computing Education researchers to better understand their perceptions, experiences, and challenges related to conducting and publishing replication studies. Our SLR identified 63 replication studies among 2,516 published papers. While the proportion of replication studies has increased slightly, overall growth remains limited. We observed a shift toward more published replication studies in journals and an increase in authors replicating their own prior work. Survey results indicate that although many researchers engage in replication within their teaching and research practice, they encounter significant challenges when attempting to publish replication studies. Despite increased discourse around open science and research rigour, the adoption of replication studies in CER has not substantially grown. Our findings offer opportunities for future research to promote replication and to explore how the CER community can encourage researchers to publish replication studies.
