Search papers, labs, and topics across Lattice.
This paper introduces the Faithful Warm-Start (FWS) strategy to enhance the reasoning capabilities of Vision-Language Models (VLMs) by curating a dataset, FaithfulQA, that emphasizes visually grounded reasoning. By leveraging explicit vision-language causal relationships from six VQA benchmarks, the approach ensures that the model is initially trained on data that promotes causal consistency and visual faithfulness. The experimental results demonstrate that this method significantly improves answer accuracy, stabilizes reinforcement learning training, and mitigates issues related to unsupported reasoning.
Faithful supervision through a novel warm-start strategy boosts VLM accuracy and stabilizes training by grounding responses in visual evidence.
Reinforcement Learning (RL) is an important paradigm for improving the reasoning capabilities of Vision-Language Models (VLMs). However, directly applying RL to rollout multimodal reasoning can lead to instability, due to the exploitation of language priors, the neglect of visual evidence, and the generation of reasoning traces that are fluent yet not visually grounded. The question arises: Can initially steer the policy toward visually faithful reasoning regime before applying reinforcement learning? To this end, we propose a Faithful Warm-Start (FWS) strategy that first curates samples with explicit vision-language causal relationships from six general VQA benchmarks to construct the FaithfulQA dataset, where each of the image-question pairs gains a certain degree of visual observations, question requirements, commonsense knowledge, domain knowledge, and the final answer. Subsequently, a VLM-based judge is employed to further purify the dataset, ensuring strong causal consistency and visual faithfulness. This warm-start stage equips the model with the capability to understand causally grounded vision-language patterns before subsequent RL optimization under sparse answer-level rewards. Experimental results show that such faithful supervision improves answer accuracy, stabilizes RL training, and reduces visually unsupported reasoning.
