Search papers, labs, and topics across Lattice.
This study introduces FutureBridge-OPD (FTB), a novel approach to on-policy distillation that evaluates teacher guidance at high-disagreement states to enhance student trajectory effectiveness. By executing a brief teacher bridge at these critical points, FTB assesses the impact of guidance on subsequent student actions, leading to improved distillation outcomes. The method was validated on multiple environments, showing an average performance increase of 16.6 points over traditional on-policy distillation methods.
Teacher guidance can be strategically enhanced by targeting high-disagreement states, leading to significant performance gains in agentic tasks.
On-policy distillation (OPD) provides teacher supervision on states visited by the student, reducing the distribution gap between training and inference. However, in multi-turn agentic tasks, student deviations may accumulate over time, gradually moving the trajectory away from states where teacher guidance remains effective. Our quantitative analysis further shows that high-disagreement states offer promising opportunities for teacher guidance, but determining whether such guidance is beneficial requires examining its effect on subsequent student trajectories. We propose FutureBridge-OPD (FTB), which executes a short teacher bridge at a high disagreement state and uses the resulting student continuation to assess whether the bridge increases the density of positive distillation signals relative to the teacher. On ALFWorld, WebShop, and ScienceWorld, under the main Qwen3-32B teacher to Qwen3-1.7B student setting, FTB outperforms vanilla OPD and TCOD by an average of 16.6 and 7.6 points, respectively, and remains effective across student scales and teacher settings. Our code is publicly available at https://github.com/ChenChiShui/FutureBridge-OPD.
