Search papers, labs, and topics across Lattice.
This paper introduces a training-free inference strategy for Multimodal Large Language Models (MLLMs) that effectively separates visual perception from logical reasoning to enhance performance. By employing a novel metric called the vision-to-text attention ratio, the method adaptively switches between latent reasoning for perceptual tokens and explicit text generation for logical tokens. Experimental results show that this approach achieves state-of-the-art performance, significantly improving both accuracy and inference efficiency while reducing latency.
The proposed Attention-Guided Switching method allows MLLMs to dynamically balance between visual fidelity and logical coherence, achieving unprecedented efficiency in reasoning tasks.
Reasoning in Multimodal Large Language Models (MLLMs) requires both fine-grained visual perception and rigorous logical deduction. Explicit text-based Chain-of-Thought (CoT) is computationally expensive and prone to visual hallucinations, while existing latent reasoning methods typically require costly training. Furthermore, directly adapting training-free LLM reasoning mechanisms to the multimodal setting yields unstable performance. We identify that this failure stems from their reliance on token-level entropy, which fundamentally conflates perceptual ambiguity (e.g., unclear visual details) with logical uncertainty (e.g., complex reasoning steps). To overcome this bottleneck, we present a novel training-free inference strategy for MLLMs that explicitly decouples perception and reasoning. We propose a novel metric, the vision-to-text attention ratio, to dynamically gauge the model's cognitive focus. Guided by this metric, our proposed framework, Attention-Guided Switching (AGS), adaptively triggers latent reasoning for perceptual tokens to preserve high-fidelity visual information in the continuous space, while enforcing explicit text generation for logical tokens to maintain structural anchoring. Extensive experiments demonstrate that our method achieves state-of-the-art performance, significantly improving both accuracy and inference efficiency by reducing autoregressive steps and latency. Code is released at https://github.com/swordAndSnow/MM26-AGS.
Tsinghua AI