Search papers, labs, and topics across Lattice.
To address the failure mode of multimodal tool-use agents hallucinating on noisy retrieved web contexts under sparse outcome supervision, this work develops Self-Verification via Reinforcement Learning (SVRL) to incentivize agents to internally validate retrieved evidence within their reasoning traces. By pairing outcome-based RL with search-cost penalties and query-diversity rewards, the framework eliminates the inference overhead of auxiliary verification models. Applying SVRL to Qwen-2.5-VL-7B on just 5,000 VQA examples yields significant multi-hop reasoning gains and search efficiency, substantially narrowing the performance gap with larger proprietary models.
Multimodal agents do not need external verifier modules to handle noisy retrieval鈥攖argeted RL rewards can teach a 7B model to internally audit search results and approach proprietary-tier multi-hop reasoning on just 5,000 training samples.
Reasoning agents increasingly rely on external tools such as web search to answer complex queries. Reinforcement learning (RL) finetuning algorithms such as GRPO have improved long-form reasoning in text-only language models, particularly for coding and mathematics. Reliable tool use in multimodal agents, however, remains challenging because models must interpret text and images while integrating noisy retrieved evidence, often under sparse outcome-level supervision without explicit verification signals. We present Self-Verification via Reinforcement Learning (SVRL), an RL-only finetuning framework that trains multimodal agents to verify and filter retrieved evidence within their own reasoning traces, reducing reliance on external verifiers at inference time. SVRL also introduces a search-aware penalty that discourages unnecessary tool calls and a query-diversity reward that encourages diverse, well-formed search queries, providing fine-grained feedback on when and what to search. Finetuning Qwen-2.5-VL-7B with SVRL on only 5{,}000 visual question answering examples yields consistent gains in multi-hop VQA generalization and tool efficiency across benchmarks. Overall, SVRL narrows the gap between compact agents and much larger proprietary models while requiring substantially lower training and inference cost.
UW