Search papers, labs, and topics across Lattice.
The paper introduces OCR-Agent, an iterative self-correction framework for VLMs that enhances their ability to perform OCR and visual reasoning tasks. It addresses the lack of self-correction mechanisms in existing VLMs by incorporating Capability Reflection for error diagnosis and correction planning, and Memory Reflection to avoid repetitive attempts. Experiments on OCRBench v2 demonstrate that OCR-Agent outperforms InternVL3-8B and achieves state-of-the-art results in visual understanding and reasoning, highlighting the benefits of structured reflection.
Forget finetuning – structured self-reflection alone can boost VLM reasoning robustness, surpassing even larger models on OCRBench v2.
Large Vision-Language Models (VLMs) have demonstrated significant potential on complex visual understanding tasks through iterative optimization methods.However, these models generally lack effective self-correction mechanisms, making it difficult for them to independently rectify cognitive biases. Consequently, during multi-turn revisions, they often fall into repetitive and ineffective attempts, failing to achieve stable improvements in answer quality.To address this issue, we propose a novel iterative self-correction framework that endows models with two key capabilities: Capability Reflection and Memory Reflection. This framework guides the model to first diagnose errors and generate a correction plan via Capability Reflection, then leverage Memory Reflection to review past attempts to avoid repetition and explore new solutions, and finally, optimize the answer through rigorous re-reasoning. Experiments on the challenging OCRBench v2 benchmark show that OCR-Agent outperforms the current open-source SOTA model InternVL3-8B by +2.0 on English and +1.2 on Chinese subsets, while achieving state-of-the-art results in Visual Understanding (79.9) and Reasoning (66.5) - surpassing even larger fine-tuned models. Our method demonstrates that structured, self-aware reflection can significantly enhance VLMs'reasoning robustness without additional training. Code: https://github.com/AIGeeksGroup/OCR-Agent.
