Search papers, labs, and topics across Lattice.
This paper introduces SHIFT, a training-free data selection method for Reinforcement Learning with Verifiable Rewards (RLVR) that operates without labels or reward evaluation on the full candidate pool. SHIFT leverages inference-time hidden-state dynamics by computing a reasoning-induced representation shift (RIRS) from a single deterministic rollout, using the RIRS magnitude as a proxy for instance utility. Experiments on mathematical reasoning and medical QA benchmarks demonstrate that SHIFT outperforms existing training-free baselines, improving both in-domain accuracy and transfer performance under ultra-low data budgets.
Forget training loops and labels: this method selects high-value reasoning examples for RL using only a single forward pass through the model.
Reinforcement learning with verifiable rewards (RLVR) can yield large reasoning gains from very few training instances, yet its strong sensitivity to which instances are used makes data selection a central bottleneck. Most existing selection pipelines rely on training-time optimization signals and/or require access to verifiable rewards or ground-truth answers over large candidate pools, which is costly and often infeasible in specialized domains. We study RLVR data selection in a setting where selection must be performed before any RL training and without labels or reward evaluation on the full pool. We propose SHIFT, a one-shot, training-free selector based solely on inference-time hidden-state dynamics. For each candidate instance, SHIFT runs a single deterministic reasoning rollout and computes a reasoning-induced representation shift (RIRS) as the start-to-end hidden-state delta. SHIFT uses the RIRS magnitude as a lightweight proxy for instance utility and enforces coverage via a quality-weighted farthest-first CoreSet procedure in an RIRS-augmented feature space, producing compact subsets that scale to large unlabeled pools. Across mathematical reasoning and medical QA benchmarks under ultra-low budgets, SHIFT consistently outperforms training-free diversity and difficulty/uncertainty baselines, improving both in-domain accuracy and transfer to harder evaluation settings. Ablations show that RIRS-based coverage and quality-weighting contribute complementary gains, and analyses indicate that RIRS is not explained by simple input/output length statistics. Code is available at github.com/JianghaoWu/SHIFT.
