Search papers, labs, and topics across Lattice.
This paper introduces the LatEnt Noise maSk (Lens), a framework designed to enhance multimodal large language models (MLLMs) by purifying visual evidence for improved fine-grained reasoning. By employing a Lens Evidence Token (LET) to identify and retain relevant visual cues while injecting adaptive latent noise into less relevant tokens, Lens effectively reduces visual redundancy without altering the model architecture. The approach yields significant performance improvements, achieving 2.4-6.4 point gains on visual question answering (VQA) datasets and 4.1-6.4 points on grounding tasks, underscoring the importance of cleaner visual inputs for multimodal reasoning.
Cleaner visual cues can boost multimodal reasoning performance by over 6 points, challenging the notion that simply extending reasoning traces is sufficient.
Multimodal large language models (MLLMs) often fail in fine-grained visual reasoning, as question-relevant visual cues are diluted by dense and redundant image tokens. Recent multimodal reasoning methods usually extend chain-of-thought from language models into visual or latent spaces, seeking to add intermediate reasoning states while overlooking the negative impact of redundant visual tokens. We propose LatEnt Noise maSk (Lens), a question-conditioned visual evidence purification framework that empowers MLLMs to reason with cleaner visual cues in latent space. Lens introduces a lightweight Lens Evidence Token (LET) to score which visual tokens support the current question and preserve them during decoding. Guided by the LET scores, it injects adaptive latent noise into low-relevance tokens, softly suppressing distractors without changing the model backbone or token sequence. With only one temporary learnable control token and a lightweight noise generator, Lens adds minimal overhead while improving the base MLLM by 2.4-6.4 points on most VQA datasets and by 4.1-6.4 points on grounding tasks. These results show that multimodal reasoning can benefit more directly from cleaner question-relevant visual evidence than from simply extending the reasoning trace.
