Search papers, labs, and topics across Lattice.
This paper introduces ReVisIT, a train-free framework that enhances multimodal in-context learning by leveraging retrieved labeled exemplar images as units of visual thought. By employing structured class definitions and per-query multimodal retrieval, ReVisIT achieves competitive performance on the VL-ICL Bench Fast Open MiniImageNet, reaching 98.5% accuracy at 4-shot, closely matching the state-of-the-art LLaVA-OneVision model while using significantly fewer parameters. The findings highlight that the retrieval quality is the primary contributor to performance gains, underscoring the potential of retrieval-based approaches in dynamic visual reasoning.
Retrieval-based visual reasoning can match state-of-the-art performance with half the parameters, revolutionizing how we think about multimodal learning.
Recent work on Thinking with Images makes vision a dynamic part of reasoning, but does so through generation: the model invokes external tools, synthesizes code, or imagines new imagery, each at the cost of a tool protocol, brittle code, or an expensive training pipeline. A fourth route makes vision dynamic without generating anything, by retrieving labeled exemplar images and reasoning over them, yet it remains underexplored despite being train-free. We present ReVisIT, a train-free framework that realizes this retrieval-based route by treating each retrieved image-label pair as a unit of visual thought. ReVisIT combines structured class definitions, per-query multimodal retrieval of exemplars, and alternating user/assistant injection of those exemplars before joint multi-attribute decoding, and degrades gracefully to whichever components a task admits. On VL-ICL Bench Fast Open MiniImageNet, Qwen3-VL-30B-A3B with ReVisIT reaches 98.5% at 4-shot, statistically indistinguishable from the 72B LLaVA-OneVision SOTA (98.7%) on this near-saturated task at about 1/2.4 the parameters, while the same backbone without the scaffold sits at chance. The turns layer alone adds 26.1 points to GPT-4.1 on free-form concept induction (Bongard-OpenWorld), and the full stack yields a 4-6 point macro gain across three backbones on MAAC-Bench, a new license-clean 27-class, 5-attribute benchmark, significant by paired bootstrap on the curator-derived attributes. Component analysis shows that retrieval-plus-turns is the universal lever while structured definitions are need-adaptive, and that 83% of the retrieval gain comes from retrieval quality rather than from the presence of exemplars. MAAC-Bench is released with a rubric-grounded LLM verification protocol that replaces author spot-check on subjective attributes.
