DICS: Exploring Data Intrinsic Consistency for Visual Instruction Selection

2026-08-30

AI Summary

为解决视觉指令调优中数据子集选择问题，提出基于数据内在一致性（DIC）的自评分指标及选择方法DICS，优化样本内一致性和全局分布多样性。

Abstract

Visual instruction tuning is crucial for advancing the vision-language alignment and instruction-following capabilities of Vision-Language Models (VLMs). However, identifying optimal subsets under a fixed ratio constraint from rapidly expanding datasets remains a significant bottleneck. While existing methods largely depend on distribution diversity or heuristic filtering, they often overlook the internal coherence within individual samples. To bridge this gap, we propose Data Intrinsic Consistency (DIC), a self-scoring metric designed to quantify the sample-level inter-component consistency. DIC consists of two modules: Visual Information Consistency (VIC), evaluating the alignment between visual content and instructions, and Response Information Consistency (RIC), assessing response coherence relative to the instruction. Building upon DIC, we introduce Data Intrinsic Consistency Selection (DICS), an adaptive data selection method that optimizes the trade-off between high intra-sample consistency and global distributional diversity under varying data budgets. Extensive experiments demonstrate that DICS consistently outperforms state-of-the-art methods across diverse dataset scales and model architectures, surpassing full-dataset fine-tuning while using only 25% of the LLaVA-1.5-665K data. We further curate DICS-6M, a 6M-sample multi-modal instruction corpus that enables the largest-scale visual instruction selection study to date; remarkably, DICS reaches 94.52\% of the official InternVL3-8B-Instruct performance using less than 25\% of its reported training data. Code can be seen at https://github.com/cqu-student/DICS

Problem

• Visual Instruction Tuning
• Data Intrinsic Consistency
• Vision-Language Models
• Dataset Selection
• Sample-Level Consistency

Innovation

• Data Intrinsic Consistency
• Visual Instruction Tuning
• Adaptive Data Selection
• Vision-Language Models