Search papers, labs, and topics across Lattice.
This paper introduces Transfer-Aware Curriculum (TAC), an innovative online curriculum for multi-domain reinforcement learning with verifiable rewards (RLVR) that dynamically prioritizes domains based on their potential to enhance overall training performance. By leveraging per-domain advantages and projected gradients to assess cross-domain transferability, TAC significantly outperforms traditional curriculum approaches, achieving the highest macro-averaged accuracy across a six-domain reasoning suite. The results demonstrate that incorporating transferability into curriculum design can lead to substantial improvements in learning efficiency and effectiveness, particularly in imbalanced training scenarios.
Prioritizing domains based on their cross-domain transferability can boost multi-domain RLVR performance by up to 10%.
Reinforcement learning with verifiable rewards (RLVR) has been extended from single-domain training to multi-domain reasoning suites spanning mathematics, programming, and science. However, the training curriculum (how often each domain is sampled) is typically fixed or hand-tuned, even though reasoning skills transfer unevenly across domains. Existing learnability-based curricula adapt to where the policy is currently improving, but are blind to whether a gradient step on the selected domain benefits the remaining domains. In this paper, we propose Transfer-Aware Curriculum (TAC), a bandit-style online curriculum that prioritizes domains whose updates broadly benefit the rest of the training suite. TAC repurposes signals already produced by RL training: per-domain advantages capture local learnability, and projected gradients, taken from the GRPO step being computed, estimate cross-domain transferability via gradient-geometry alignment, at negligible cost (<1% wall-clock overhead). Across a six-domain reasoning suite, TAC achieves the best macro-averaged accuracy on both Qwen3-1.7B and Llama3.2-3B, outperforming proportional random sampling, a hand-designed schedule, and a learnability-only bandit, and improving over the last of these by up to 2.8 points (10% relative). Ablations show performance degrades sharply when the transferability term is removed, and TAC remains robust on imbalanced training mixtures where learnability-only curricula over-commit to dominant domains. Our findings establish cross-domain transferability as a key signal for curriculum design in multi-domain RLVR.
