Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper investigates how multimodal reinforcement learning with verifiable rewards (RLVR) learns to utilize visual information, finding that models trained without images still recover significant performance gains when images are introduced at test time. Prolonged real-image training can paradoxically degrade visual grounding while maintaining benchmark scores, highlighting a disconnect between image presence and learning signal. The authors introduce "visual resolvability" as a design rule, requiring visual evidence to be necessary and the task to remain learnable, demonstrating substantial improvements in visual target discovery and generalization with a 7B model.
Multimodal RL models trained without visual input can still achieve high accuracy on visual reasoning tasks when images are present at inference, but prolonged visual training can paradoxically erode grounding.
Reinforcement learning with verifiable rewards (RLVR) improves vision-language benchmark scores even without visual information during training. With images at test, blind-trained models recover roughly half of the real-image gain at 3B and nearly four fifths at 7B. Prolonged real-image training can erode grounding while benchmark gains persist. Both findings expose the same gap: an image in the prompt is not an image in the learning signal. Our design rule, visual resolvability, asks that visual evidence be necessary for a correct answer and that the task remain learnable. We test it on counterfactual coordinate scenes in which the question stays fixed and the target is never named, so a correct answer requires finding the target in the image. With standard GRPO and correctness-and-format rewards, a 7B model raises its accuracy at finding the target (discovery) from 0.425 to 0.875 on held-out scenes denser than any it trained on, and it improves on question types it never trained on. Two controls locate the source of the gain. Replacing test images with gray canvases drops discovery to zero; training on gray canvases instead, at matched step 30 and in each of four seeds, yields essentially none of the gain even when the model is then tested with real images. The learned skill carries over to grounding tasks built independently of the training corpus. A caption that answers the training question, added to the same images, reward and budget, cuts the gain by nearly two thirds. Changing what reward requires changes what RL learns.