Search papers, labs, and topics across Lattice.
This paper introduces ReasonCLIP-58M, a continual pretraining framework that enhances CLIP-style models by integrating large-scale reasoning supervision through a two-stage strategy. The authors construct two complementary datasets鈥擱easonLite-42M and ReasonPro-16M鈥攁nd a benchmark, RCLIP-Bench, to evaluate the models' performance in visually grounded commonsense reasoning and compositional reasoning. The results show that ReasonCLIP significantly improves reasoning capabilities and zero-shot retrieval performance, demonstrating that structured reasoning supervision can enhance the expressive capacity of visual representations without increasing inference costs.
ReasonCLIP-58M shows that structured reasoning supervision can dramatically boost CLIP's reasoning abilities while maintaining efficiency.
CLIP and its variants are widely adopted visual backbones in multimodal systems, but their pretraining remains dominated by descriptive image-text alignment. As downstream applications increasingly demand visually grounded commonsense inference and compositional reasoning, it remains unclear whether CLIP-style encoders can support such reasoning without architectural changes. To address this, we present ReasonCLIP-58M, a continual pretraining framework that integrates large-scale reasoning supervision into CLIP-style models through our two-stage strategy, which progressively integrates reasoning signals while preserving descriptive alignment, followed by category-structured reasoning supervision. To support this framework, we construct two complementary datasets and a benchmark: ReasonLite-42M, with open-form, visually verifiable reasoning captions; ReasonPro-16M, with category-specific reasoning supervision; and RCLIP-Bench for diagnostic evaluation of visually grounded reasoning. We train a family of ReasonCLIP that improves visually grounded commonsense and compositional reasoning while also enhancing zero-shot retrieval performance. As a drop-in visual encoder for multimodal large language models such as LLaVA-NeXT, ReasonCLIP delivers consistent gains without additional inference cost, demonstrating that structured reasoning supervision enhances the expressive capacity of CLIP-style visual representations. All datasets, models, and training code are available at https://github.com/RISys-Lab/ReasonCLIP.
