Search papers, labs, and topics across Lattice.
DE-Venus introduces a data-efficient framework for reinforcement learning with verifiable rewards (RLVR) tailored for large language models, addressing the high costs associated with on-policy rollouts and reliable target acquisition. By organizing the data lifecycle into three modules鈥擜ctive Data Selection, Weak Supervision Construction, and Training-Time Supervision Refinement鈥攖he framework enhances model training efficiency while maintaining quality. Experimental results demonstrate that DE-Venus can achieve comparable or improved model performance using only 10% of labels or as little as 13% of relevant data, significantly reducing convergence steps by up to 75%.
Achieving high-quality model performance with just 10% of the required labels could revolutionize the scalability of RLVR in large language models.
Reinforcement learning with verifiable rewards (RLVR) improves large language model reasoning, but its practical scaling is constrained by expensive on-policy rollouts and the cost of obtaining reliable targets at scale. Existing methods address sample selection, incomplete supervision, or noisy labels separately, often entangling supervision logic with distributed training and hindering controlled comparison and reuse. We present DE-Venus, a unified framework for data-efficient RLVR that treats supervision as evolving state across data preparation and policy optimization. It organizes this lifecycle into three modules: Active Data Selection allocates training and annotation budgets; Weak Supervision Construction derives learning signals from unlabeled examples; and Training-Time Supervision Refinement filters or corrects unreliable supervision. DE-Venus supports seven representative methods and a data-selection pipeline by expressing method-specific decisions as offline dataset transitions or online transformations of targets, rewards, batches, and advantages while preserving verl's distributed execution contracts. Across public benchmarks and three business scenarios, separate configurations preserve or improve model quality with only 10% of labels or as little as 13% of relevant data; selected business configurations also reduce observed convergence steps by 63%--75%. DE-Venus thus reduces annotation and training costs without sacrificing scalable RL execution.
