Search papers, labs, and topics across Lattice.
This paper introduces VisReflect, a novel framework that enhances fine-grained perception in long visual contexts by utilizing latent visual reflection instead of traditional discrete token decoding. By generating continuous visual reflections that emphasize salient features, the model effectively guides attention towards relevant visual tokens in a single forward pass, mitigating the visual attention sink phenomenon. Comprehensive evaluations demonstrate significant performance improvements over strong baselines, with gains of 4.1% on image benchmarks and 1.8% on video benchmarks, while also reducing inference time by approximately 44% compared to existing zooming-based methods.
Latent visual reflection allows models to achieve superior fine-grained perception in long visual contexts while slashing inference time by nearly half.
Large Vision Language Models (LVLMs) have achieved remarkable success on vision-language tasks, yet fine-grained perception over high-resolution images and long-context videos remains challenging. As the number of visual tokens increases, the visual attention sink phenomenon becomes increasingly severe, causing irrelevant tokens to absorb a disproportionate amount of attention mass. Recent approaches attempt to mitigate this issue by explicitly predicting bounding boxes or temporal spans and re-encoding the cropped visual regions. Such methods depend on unreliable numeric localization in the discrete token space and incur significant computational overhead due to additional forward passes. In this work, we propose **VisReflect**, a simple yet effective framework that improves fine-grained perception in long visual contexts through latent visual reflection. Instead of decoding intermediate predictions into discrete tokens, the model generates continuous visual reflection that represents question-relevant visual features in the latent space. These reflections selectively emphasize salient regions or frames, guiding attention towards relevant visual tokens within a single forward pass. We conduct comprehensive evaluations on challenging high-resolution image benchmarks, including BLINK, V*, and HRBench-4K/8K, as well as video understanding benchmarks such as MVBench, VideoMME, and MLVU. Our method consistently improves over strong baselines, achieving gains of 4.1% on image benchmarks and 1.8% on video benchmarks. Compared with zooming-based methods, our model achieves comparable performance while reducing inference time by roughly 44% on video understanding.
