Search papers, labs, and topics across Lattice.
This paper introduces Timage, a novel approach that enhances fine-tuning of vision-language models by overlaying textual queries directly onto images, thereby improving spatial reasoning. Utilizing a Constrained Schr枚dinger Bridge for layout synthesis, Timage effectively aligns text with relevant image regions while maintaining visual coherence and legibility. The method outperforms larger proprietary models and parameter-tuned baselines on the VMCBench suite, highlighting the efficacy of input reconstruction in multimodal understanding.
Timage transforms the way we align text and images, achieving superior multimodal reasoning with a modest model size that outperforms larger competitors.
Multimodal Large Language Models (MLLMs) often lose track of the right image regions during fine-grained spatial reasoning, because a textual query rarely carries any explicit geometric anchor into the pixel domain. Prevailing remedies either rewire the model's weights or pad the prompt with verbose instructions, yet neither reliably pins the language to the correct visual coordinates without eroding the backbone's general competence. We introduce Timage, a paradigm that recasts multimodal understanding as an alignment problem solved at the input: the query is drawn, as a typeset overlay, onto the image itself. The placement and appearance of this overlay are produced by a Constrained Schr枚dinger Bridge (cSB), an entropic optimal-transport sampler that factorizes layout synthesis into two coupled stochastic stages. The first stage, Region Search, transports noise toward query-aligned image zones while obeying a hard occlusion barrier that protects salient foreground content; the second stage, Appearance Shaping, sizes the glyphs through an ``ink-budget'' regularizer so that the rendered text stays legible and visually balanced. The resulting overlay behaves as an explicit attention beacon that channels the model's focus along spatial semantics. On the VMCBench suite, Timage paired with a modest 7B backbone clearly overtakes far larger proprietary systems as well as parameter-tuned baselines. The study positions deliberate input reconstruction as a powerful, architecture-neutral lever for strengthening multimodal reasoning.
