Search papers, labs, and topics across Lattice.
This paper introduces SIVA-RL, a novel framework for multimodal reinforcement learning that enhances visual alignment by employing sample-wise, outcome-conditioned supervision instead of traditional operator-conditioned regularization. By utilizing localized interventions through a method called PatchSwap, SIVA-RL effectively aligns sensitivity and invariance in model responses, leading to significant performance improvements across various reasoning benchmarks. The results show that SIVA-RL outperforms existing RL baselines, achieving up to a 14.9% relative improvement in overall performance and an 8.79 percentage-point gain in vision-dependent reasoning tasks.
SIVA-RL reveals that outcome-conditioned supervision can dramatically enhance multimodal reasoning performance, outperforming traditional methods across diverse benchmarks.
Reinforcement learning with verifiable rewards (RLVR) drives multimodal reasoning, but answer-level correctness does not guarantee that a vision-language model grounds its predictions in visual evidence. Existing visual-intervention methods contrast policy behavior on original and modified images, yet assign supervision by the type of intervention rather than its observed effect. This assumption fails: identical operators produce heterogeneous outcomes across samples. We propose SIVA-RL, a Sensitivity-Invariance Visual Alignment framework that replaces operator-conditioned regularization with sample-wise, outcome-conditioned supervision. SIVA-RL constructs localized interventions through token-aligned, distance-constrained within-image PatchSwap. A frozen audit policy then scores each clean-intervention pair, and the observed reward drop becomes soft routing weights. Large-drop pairs drive sensitivity alignment, low-drop pairs drive clean-anchored invariance alignment, and ambiguous pairs are down-weighted. This design decouples intervention construction from supervision assignment and is compatible with both GRPO and DAPO backbones. Across nine multimodal reasoning benchmarks spanning mathematical, logical, and vision-dependent tasks, SIVA-RL improves 3B and 7B models over matched RL baselines in every setting. It yields an 8.79 percentage-point gain on vision-dependent reasoning and up to 14.9% relative overall improvement across all four GRPO- and DAPO-based configurations.
