Search papers, labs, and topics across Lattice.
To resolve the teacher quality bottleneck in on-policy distillation without relying on external models or privileged context, RISE constructs a synthetic teacher by extrapolating the update trajectory between the active RLVR checkpoint and a trailing anchor in parameter or logit space. This mechanism recursively transforms sparse, outcome-based verifiable rewards into dense token-level distillation targets that evolve alongside the student. Across mathematical reasoning, STEM, code generation, and agentic benchmarks, the recursive distillation loop consistently outperforms both standalone RLVR training and standard on-policy self-distillation.
Models can autonomously bootstrap their own dense token-level supervision simply by extrapolating the trajectory of their own RL updates away from a trailing checkpoint.
On-policy distillation (OPD) provides dense, per-token supervision for language model post-training, but its effectiveness is bottlenecked by teacher quality: external teachers suffer from distribution mismatch, while self-distillation with privileged conditioning is limited by in-context learning capacity. We propose \textbf{RISE} (\textbf{R}ecursive \textbf{I}mprovement via \textbf{S}elf-\textbf{E}xtrapolating Policy Distillation), which constructs a synthetic teacher directly from the model's own RLVR training trajectory. By extrapolating the displacement between the current checkpoint and a trailing anchor---in parameter space or output logit space---RISE converts a sparse outcome-induced parameter update into a dense token-level target, without any external model or privileged conditioning. RISE combines RLVR and OPD in a complementary loop: outcome rewards ground the extrapolation toward correct reasoning, while the extrapolated teacher refines token-level decisions. Moreover, since the teacher is refreshed every iteration as the student improves, distillation becomes a recursive improvement mechanism rather than a one-shot compression step. Experiments spanning mathematical reasoning, multi-domain STEM, code generation, and multi-turn agentic tasks show that RISE outperforms RLVR-only training and on-policy self-distillation across all settings.
