Search papers, labs, and topics across Lattice.
This paper introduces Data Intrinsic Consistency (DIC), a self-scoring metric that evaluates the coherence of visual content and instructions in Vision-Language Models (VLMs), addressing the challenge of optimal data selection from large datasets. By implementing DIC within the Data Intrinsic Consistency Selection (DICS) framework, the authors achieve a balance between high intra-sample consistency and global distribution diversity, significantly improving data selection efficiency. Experimental results show that DICS outperforms existing methods, achieving 94.52% of the performance of a larger model while utilizing only 25% of the training data, thereby setting a new benchmark in visual instruction tuning.
Achieving 94.52% of state-of-the-art performance with just 25% of the training data reveals a groundbreaking efficiency in visual instruction tuning.
Visual instruction tuning is crucial for advancing the vision-language alignment and instruction-following capabilities of Vision-Language Models (VLMs). However, identifying optimal subsets under a fixed ratio constraint from rapidly expanding datasets remains a significant bottleneck. While existing methods largely depend on distribution diversity or heuristic filtering, they often overlook the internal coherence within individual samples. To bridge this gap, we propose Data Intrinsic Consistency (DIC), a self-scoring metric designed to quantify the sample-level inter-component consistency. DIC consists of two modules: Visual Information Consistency (VIC), evaluating the alignment between visual content and instructions, and Response Information Consistency (RIC), assessing response coherence relative to the instruction. Building upon DIC, we introduce Data Intrinsic Consistency Selection (DICS), an adaptive data selection method that optimizes the trade-off between high intra-sample consistency and global distributional diversity under varying data budgets. Extensive experiments demonstrate that DICS consistently outperforms state-of-the-art methods across diverse dataset scales and model architectures, surpassing full-dataset fine-tuning while using only 25% of the LLaVA-1.5-665K data. We further curate DICS-6M, a 6M-sample multi-modal instruction corpus that enables the largest-scale visual instruction selection study to date; remarkably, DICS reaches 94.52\% of the official InternVL3-8B-Instruct performance using less than 25\% of its reported training data. Code can be seen at https://github.com/cqu-student/DICS
DAMO