Search papers, labs, and topics across Lattice.
This paper investigates the concept of task learnability as a static prior for optimizing reinforcement learning (RL) in large language models (LLMs), addressing the limitations of uniform task sampling that neglects task-specific optimization responses. By analyzing per-task reward trajectories, the authors establish that task learnability is a reproducible metric that predicts downstream utility, thus enabling more efficient training. The proposed method, TrajVal, serves as a lightweight estimator for task learnability and demonstrates improved data efficiency and complementary benefits when integrated with existing online scheduling methods across various reasoning benchmarks.
Task learnability can significantly enhance RL efficiency in LLMs, leading to better performance with less data.
Reinforcement learning (RL) has become a central post-training paradigm for eliciting reasoning capabilities in large language models, yet uniform task sampling allocates compute without regard to differences in how tasks respond to optimization. Existing task-valuation methods mostly rely on snapshot-based signals such as current pass rate or reward, which estimate how solvable a task is under the current policy. However, tasks with similar current solvability can still differ substantially in how positively they respond to further training. We study this residual axis as task learnability: a regime-conditional measure of expected positive response to continued training under a fixed RL post-training regime. By analyzing per-task reward trajectories, we find that learnability is reproducible across independently sampled training contexts and predictive of downstream utility. To make this signal practical before training begins, we propose TrajVal, a lightweight probe-based estimator that approximates per-task learnability from a short probe run and two endpoint evaluations. TrajVal can be used either as a standalone static prior for task sampling or as a multiplicative prior for existing online schedulers. Experiments on mathematical and logical reasoning benchmarks across multiple model scales show that TrajVal improves data efficiency over uniform sampling and provides complementary gains when combined with online scheduling methods.
DAMO