Search papers, labs, and topics across Lattice.
This paper introduces a trajectory-guided structured sampling approach to align large vision-language models (LVLMs) at test time, addressing the limitations of traditional post-training reinforcement learning methods that are often resource-intensive and misaligned with inference-time distributions. By curating a reasoning memory bank and employing an iterative Markov Chain Monte Carlo (MCMC) algorithm, the authors achieve enhanced visual grounding and logical consistency during inference. Experimental results across various multimodal reasoning datasets show a significant accuracy improvement without the high inference costs typically associated with alignment techniques.
Trajectory-guided test-time sampling can enhance LVLM accuracy while sidestepping the resource burdens of traditional alignment methods.
Post-training reinforcement learning (RL) algorithms are commonly used to align large vision-language models (LVLMs) with human intent and the requirements of visual reasoning tasks. However, existing RL-based alignment methods are often resource-intensive and encounter mismatches between training objectives and inference-time distributions. To bridge this gap, we propose a novel test-time alignment approach that leverages trajectory-guided structured sampling for dynamic inference-time refinement, achieving better alignment with visual grounding and ensuring logical consistency. Our approach begins with curating a reasoning memory bank via a trajectory learning algorithm, which decomposes complex question solving into ordered sequences of predefined reasoning patterns. It subsequently accomplishes inference-time alignment by first collecting trajectories from reasoning memory bank to establish a global structural reasoning prior, and then using an iterative Markov Chain Monte Carlo (MCMC) algorithm for localized multi-objective refinement of the reasoning trace. Experiments across multiple multimodal reasoning datasets demonstrate that our approach significantly improves accuracy without incurring prohibitive inference overhead. These results establish trajectory-guided test-time sampling as a scalable and effective alternative to traditional post-training alignment, particularly for complex visual reasoning tasks.
