Search papers, labs, and topics across Lattice.
This paper introduces CVPD (Contrastive Counterfactual Visual Process Distillation), a novel framework for self-contained, token-level visual self-distillation in multimodal large language models (MLLMs). By identifying visual blind spots where model responses vary based on localized context, CVPD generates dense contrastive supervision directly from the model's own outputs, enhancing its ability to leverage perceptual information. The method demonstrates significant improvements over existing self-evolving baselines, achieving notable gains across multiple benchmarks without any regressions in performance.
Visual blind spots can be transformed into powerful self-supervision signals, leading to substantial performance gains in multimodal language models.
Self-improvement for multimodal large language models (MLLMs) is typically driven by reward-based methods that provide only coarse scalar feedback. Distillation offers a richer alternative through dense token-level supervision, but in the visual domain it usually depends on privileged context constructed using external annotations and tools, or stronger models. We introduce \textbf{CVPD} (Contrastive Counterfactual Visual Process Distillation), which, to the best of our knowledge, is the first fully self-contained framework for dense, on-policy, token-level visual self-distillation for MLLMs. CVPD identifies visual blind spots where zooming into a region changes and sharpens the model's answer distribution, while removing the same region leaves the full-image behavior largely unchanged. Such regions reveal perceptual information that the model can encode but fails to consistently utilize under full-image conditioning. We propose a three-gate Counterfactual Criterion that identifies these regions directly from the model's own responses and converts them into dense contrastive supervision for self-distillation. On Qwen3-VL-8B-Instruct, CVPD outperforms six self-evolving baselines across twelve benchmarks, including methods that rely on external GPT-4o supervision, without a single regression. It achieves gains of $+3.60$ on OCRBench, $+3.38$ on MMStar Fine-Grained Perception, and $+3.08$ on MMStar Logical Reasoning, while maintaining or improving performance on broader multimodal benchmarks.
