Search papers, labs, and topics across Lattice.
This paper introduces ICLE++, a new corpus of persuasive student essays that includes both holistic and trait-specific scores, addressing the limitations of existing models evaluated solely on the ASAP corpus. The authors highlight the importance of generalizability in automated essay scoring (AES) by providing a resource that enables testing across different corpora and supports emerging AES challenges like multi-trait and cross-prompt scoring. Key findings indicate that ICLE++ can significantly enhance the evaluation of AES models, fostering advancements in the field.
ICLE++ reveals that existing AES models may struggle with generalization, highlighting the need for diverse evaluation datasets.
The majority of the recently developed models for automated essay scoring (AES) are evaluated solely on the ASAP corpus. However, ASAP is not without its limitations. For instance, it is not clear whether models trained on ASAP can generalize well when evaluated on other corpora. In light of these limitations, we introduce ICLE++, a corpus of persuasive student essays annotated with both holistic scores and trait-specific scores. Not only can ICLE++ be used to test the generalizability of AES models trained on ASAP, but it can also facilitate the evaluation of models developed for newer AES problems such as multi-trait scoring and cross-prompt scoring. We believe that ICLE++, which represents a culmination of our long-term effort in annotating the essays in the ICLE corpus, contributes to the set of much-needed annotated corpora for AES research.
