Search papers, labs, and topics across Lattice.
This paper introduces a new task, Scoring Edit Impact in Grammatical Error Correction (GEC), to automatically estimate the importance of edits produced by GEC systems. They propose a scoring framework based on an embedded association graph that captures latent dependencies among edits and syntactically related edits. Experiments across 4 GEC datasets, 4 languages, and 4 GEC systems demonstrate that their method consistently outperforms a range of baselines in estimating edit importance.
Accurately scoring the impact of individual edits in grammatical error correction is now possible without human annotation, thanks to a novel graph-based approach.
A Grammatical Error Correction (GEC) system produces a sequence of edits to correct an erroneous sentence. The quality of these edits is typically evaluated against human annotations. However, a sentence may admit multiple valid corrections, and existing evaluation settings do not fully accommodate diverse application scenarios. Recent meta-evaluation approaches rely on human judgments across multiple references, but they are difficult to scale to large datasets. In this paper, we propose a new task, Scoring Edit Impact in GEC, which aims to automatically estimate the importance of edits produced by a GEC system. To address this task, we introduce a scoring framework based on an embedded association graph. The graph captures latent dependencies among edits and syntactically related edits, grouping them into coherent groups. We then perform perplexity-based scoring to estimate each edit's contribution to sentence fluency. Experiments across 4 GEC datasets, 4 languages, and 4 GEC systems demonstrate that our method consistently outperforms a range of baselines. Further analysis shows that the embedded association graph effectively captures cross-linguistic structural dependencies among edits.
