Search papers, labs, and topics across Lattice.
This paper investigates the internal dynamics of vision-language models (VLMs) to address the instability of visual evidence in evidence-grounded reasoning. By identifying a three-stage process of multimodal attention focus, particularly emphasizing the Visual Relay Window (VRW), the authors reveal how task demands influence attention allocation and reasoning quality. The proposed TRACE framework enhances this process, yielding significant improvements in grounding-sensitive tasks across multiple VLM architectures and benchmarks, with an average gain of 4.33 points.
Task-adaptive control of multimodal focus can boost evidence-grounded reasoning performance by over 6 points in critical scenarios.
Vision-language models increasingly succeed on multimodal reasoning benchmarks, yet their visual evidence often becomes unstable once it enters the language stack, weakening evidence-grounded reasoning. To understand this fragility, we examine the internal dynamics of VLMs through a mechanistic lens and uncover a stable three-stage redistribution of multimodal attention focus across depth: an early question-conditioned organization, a critical middle visual-dominant relay, and a late return to answer formation. We operationalize the middle phase as the Visual Relay Window (VRW), and show that its geometry varies with task demand, is causally tied to grounded generation, and distinguishes unsupported answers from stronger reasoning trajectories. Guided by this internal rhythm, we propose TRACE, a task-adaptive inference-time control framework with lightweight trained modules. It reshapes relay allocation during prefill and preserves assembled visual support after handoff during decoding. Across four open-weight VLM backbones and seven benchmarks, TRACE delivers large gains on grounding-sensitive settings, improving them by 4.33 points on average and by up to 6.6 points, while also improving reasoning-heavy tasks. These results show that explicitly controlling multimodal focus across depth offers a unified and effective mechanism for strengthening evidence-grounded multimodal reasoning.
