Search papers, labs, and topics across Lattice.
Q-CueGraph introduces a novel approach for multimodal reasoning by explicitly mapping questions and image representations to targeted, budgeted observations, enhancing the decision-making process for image inspection. This method leverages a reusable OCR/layout graph for text-rich images and employs query-conditioned visual nodes for natural-image search, optimizing the selection and composition of visual evidence. The results demonstrate a significant accuracy improvement, achieving 0.833 on V*Bench with only 19% of the image area, compared to 0.696 using full-image inference, highlighting the effectiveness of localized observation in multimodal tasks.
Query-conditioned visual evidence graphs can boost multimodal reasoning accuracy by over 20% while using significantly less image data.
High-resolution pixels and crop or zoom tools give multimodal large language models the ability to inspect an image, but they do not provide a reliable task-conditioned policy for deciding where to inspect. Q-CueGraph makes this decision explicit. It maps a question and an image representation to budgeted, coordinate-level observations for a frozen reader. Text-rich images use a reusable OCR/layout graph; natural-image search instantiates query-conditioned visual nodes behind the same selection, composition, and budgeting interface. Optional utility refinement learns which candidate crops the frozen reader can use from training-answer correctness, without region-box supervision. With a frozen Qwen2.5-VL-7B reader, Q-CueGraph reaches 0.833 accuracy on V*Bench versus 0.696 for full-image inference from a 19% image-area budget, and reaches 92% of full-image ANLS on InfographicVQA from about half the image area. Across six benchmarks, explicit observation is most valuable when evidence is localizable, the question discriminates its location, and resolution limits full-image reading.
