Search papers, labs, and topics across Lattice.
This paper introduces a fine-grained taxonomy for curriculum learning (CL) in natural language processing (NLP), distinguishing between difficulty evaluation and training scheduling to enhance systematic analysis. By identifying the perspectival nature of difficulty and formalizing CL schedulers based on expected training contributions, the authors reveal a systematic incomparability problem in existing CL research. The taxonomy not only clarifies the confusion in prior works but also provides a framework for better design and evaluation of CL strategies, ultimately facilitating more coherent evidence accumulation in the field.
Existing curriculum learning research in NLP suffers from a systematic incomparability problem, conflating distinct notions of difficulty and scheduling that stifle progress.
Despite more than a decade of curriculum learning (CL) research in NLP, the field lacks a principled account of which difficulty function or scheduler to use for a given problem. To understand what has hindered progress towards this account, we propose a fine-grained taxonomy separating difficulty evaluation from training scheduling to enable systematic analysis of CL strategies. For difficulty evaluation, we distinguish attribution source and task dependence, revealing difficulty as a perspectival concept encoding different assumptions about what makes an instance hard to learn. For scheduling, we provide the first formalisation of CL schedulers in terms of expected training contribution, enabling comparison across implementations by introducing retention regimes and monotonicity properties. Applied in a dedicated analysis of CL works in NLP, our taxonomy reveals a systematic incomparability problem: prior works conflate distinct notions of difficulty and scheduling, often pursuing different objectives under the same CL label -- hindering comparison and the accumulation of a coherent evidence base. Beyond diagnosis, the taxonomy supports the design, analysis, and comparison of CL strategies, and motivates evaluation practices that disentangle the sources of observed improvement.
