Search papers, labs, and topics across Lattice.
This paper introduces RP-OPSD, a novel approach to On-Policy Self-Distillation that leverages the information gap between high- and low-resolution images to enhance the training of multimodal large language models. By allowing a student model to learn from a teacher model that operates on higher-resolution inputs while generating trajectories at a lower resolution, the method effectively improves the student鈥檚 predictive performance without the need for additional human annotations or external models. Experimental results demonstrate a 5.45% relative performance improvement at original resolution and a 1.78x training speedup compared to traditional OPSD methods, highlighting the efficacy of resolution differences as a source of privileged information.
Leveraging resolution differences can yield significant performance gains in multimodal large language models without the need for external supervision or annotations.
On-Policy Self-Distillation (OPSD) uses privileged information available only to the teacher to provide dense token-level supervision on trajectories generated by the student. However, existing methods often rely on verified solution traces, explanations generated by external models, or manually localized visual evidence, which limits their scalable application to multimodal large language models. To address this issue, we exploit the information gap between high- and low-resolution views of the same image and propose RP-OPSD (Resolution-Privileged On-Policy Self-Distillation for Multimodal Large Language Models). During training, the student policy generates on-policy trajectories from images at one-quarter of the original resolution, while the teacher policy provides supervision using the original-resolution images. By minimizing the divergence between their output distributions along the student trajectories, the student learns the predictive behavior of the teacher under high-resolution inputs, thereby strengthening its low-resolution capability and transferring the learned improvement to original-resolution inference. RP-OPSD requires neither additional human annotations nor external models to generate solution traces but only image--question pairs. Experiments on Qwen3.5-9B show that RP-OPSD achieves a 5.45\% relative improvement in average performance at the original resolution and a $1.78\times$ training speedup over OPSD. These results demonstrate that resolution differences can serve as a simple and scalable source of privileged information, providing an effective and efficient approach to on-policy self-distillation for multimodal large language models.
Tsinghua AI