Search papers, labs, and topics across Lattice.
This paper introduces Brain-inspired Unsupervised Self-reflection (BUS), a novel training framework that enhances the reflective reasoning capabilities of Vision-Language Models (VLMs) without the need for annotated data. By leveraging the concept of backward prediction, akin to human cognitive processes, BUS allows VLMs to learn from unlabeled data while providing explicit learning signals that improve performance on complex visual tasks. Extensive experiments across eight benchmarks reveal that BUS significantly outperforms baseline models, highlighting the importance of backward prediction in VLM reasoning.
VLMs can achieve substantial improvements in reasoning performance using only unlabeled data through a novel self-reflective training framework inspired by human cognition.
Current Vision-Language Models (VLMs) often struggle to handle complex visual tasks that require consistent and fine-grained reasoning. Recent methods aim to train models to facilitate self-reflective reasoning, i.e., reviewing and improving the generated reasoning. However, they require large volumes of annotated data and lack explicit reflective behavior during test time. This work aims to bridge this gap through inspiration from neuroscience. The human brain exhibits efficient backward prediction, i.e., predicting which current states are likely to precede a given future state. In this work, we first verify that mainstream VLMs can perform backward prediction, similar to the human brain. Then, we propose Brain-inspired Unsupervised Self-reflection (BUS), a label-free training framework to enhance reflective reasoning capability in challenging image analysis. BUS enables VLMs to perform backward prediction and provide explicit learning signals on data without ground-truth labels. In this way, BUS eliminates reliance on annotated data while improving reasoning performance. Notably, BUS is compatible with popular fine-tuning methods, such as Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL). Finally, extensive experiments on 8 benchmarks demonstrate the effectiveness of BUS across a wide range of complex visual tasks. It achieves notable improvements over the base model while using only unlabeled training data. Our experimental findings validate that backward prediction capability is critical for VLM reasoning.
