Search papers, labs, and topics across Lattice.
The Self-Verified Reasoner (SVR-R1) introduces a novel multi-turn reinforcement learning framework that utilizes a model's own verification process as a learning signal to enhance multimodal reasoning. By allowing the model to self-assess its answers and engage in a second-chance rethink when it issues a negative verdict, SVR-R1 significantly improves accuracy on vision-language reasoning benchmarks compared to standard GRPO baselines. Notably, the training dynamics reveal a reduction in verification reliance while achieving higher test accuracy, indicating that the model effectively internalizes self-correction over time.
Self-verification in SVR-R1 not only boosts multimodal reasoning accuracy but also reduces the need for extensive verification, showcasing a path to more confident model outputs.
We introduce Self-Verified Reasoner (SVR-R1), a multi-turn RL framework that turns a model's own verification into a learning signal for multimodal reasoning. For each query, the model proposes an answer using the same weights, and issues a binary self-verdict (Yes/No). A'No'triggers a second-chance rethink; a'Yes,'or a turn cap, finalizes the output for computing the outcome-based reward. SVR-R1 is implemented with GRPO and an asynchronous multi-turn rollout framework and needs no external supervision or auxiliary critics. We evaluate SVR-R1 on vision-language reasoning benchmarks and show that it improves accuracy by a large margin over strong standard GRPO baselines. Training dynamics show decreasing reliance on verification-fewer verification turns, yet higher test accuracy-indicating that the gap between verification and generation narrows as the policy internalizes self-correction and chooses the most confident answer via our framework. SVR-R1 bridges the less explored intersection of inference-time self-refinement and RL training for VLMs, offering a simple yet effective recipe for bootstrapping multimodal reasoning. We will open-source \textbf{SVR-R1} to facilitate future research in VLMs.
