Search papers, labs, and topics across Lattice.
The paper introduces Wiki-R1, a data-generation-based curriculum reinforcement learning framework designed to improve multimodal large language models (MLLMs) for Knowledge-Based Visual Question Answering (KB-VQA). Wiki-R1 employs controllable curriculum data generation to create training samples of varying difficulty by manipulating the retriever and uses a curriculum sampling strategy to select informative samples for RL updates. Experiments on Encyclopedic VQA and InfoSeek benchmarks demonstrate state-of-the-art results, with accuracy improvements of 1.6% and 4.0% respectively, showcasing the effectiveness of incentivizing reasoning through curriculum learning.
Curriculum reinforcement learning closes the distributional gap between pre-trained MLLMs and KB-VQA, yielding SOTA results by strategically generating and sampling training data.
Knowledge-Based Visual Question Answering (KB-VQA) requires models to answer questions about an image by integrating external knowledge, posing significant challenges due to noisy retrieval and the structured, encyclopedic nature of the knowledge base. These characteristics create a distributional gap from pretrained multimodal large language models (MLLMs), making effective reasoning and domain adaptation difficult in the post-training stage. In this work, we propose \textit{Wiki-R1}, a data-generation-based curriculum reinforcement learning framework that systematically incentivizes reasoning in MLLMs for KB-VQA. Wiki-R1 constructs a sequence of training distributions aligned with the model's evolving capability, bridging the gap from pretraining to the KB-VQA target distribution. We introduce \textit{controllable curriculum data generation}, which manipulates the retriever to produce samples at desired difficulty levels, and a \textit{curriculum sampling strategy} that selects informative samples likely to yield non-zero advantages during RL updates. Sample difficulty is estimated using observed rewards and propagated to unobserved samples to guide learning. Experiments on two KB-VQA benchmarks, Encyclopedic VQA and InfoSeek, demonstrate that Wiki-R1 achieves new state-of-the-art results, improving accuracy from 35.5\% to 37.1\% on Encyclopedic VQA and from 40.1\% to 44.1\% on InfoSeek. The project page is available at https://artanic30.github.io/project_pages/WikiR1/.
