InstructDiff: Domain-Adaptive Data Selection via Differential Entropy for Efficient LLM Fine-Tuning

2026-01-30

AI Summary

This work addresses the limited generalization of existing data selection methods for supervised fine-tuning of large language models in cross-domain scenarios, where trade-offs between efficiency and performance remain challenging. It reveals, for the first time, a directional relationship between differential entropy shifts and task domains: reasoning tasks favor entropy increase (cognitive expansion), whereas general instruction-following tasks prefer entropy decrease (cognitive compression). Building on this insight, the authors propose a unified domain-adaptive data selection framework that integrates warm-up calibration, bidirectional negative log-likelihood filtering, and differential entropy-based ranking to enable efficient, task-aware data curation. Using only 10% of the training data, the method outperforms full-data training by 17% on mathematical reasoning tasks and achieves a 52% improvement on general instruction-following benchmarks, substantially surpassing current baselines.

Technology Category

• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Natural Language Processing: (Large) Language Models
• Application Domains: Humanities & Computational Social Science

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling

Abstract

Supervised fine-tuning (SFT) is fundamental to adapting large language models, yet training on complete datasets incurs prohibitive costs with diminishing returns. Existing data selection methods suffer from severe domain specificity: techniques optimized for general instruction-following fail on reasoning tasks, and vice versa. We observe that measuring entropy differences between base models and minimally instruction-tuned calibrated models reveals a pattern -- samples with the lowest differential entropy consistently yield optimal performance across domains, yet this principle manifests domain-adaptively: reasoning tasks favor entropy increase (cognitive expansion), while general tasks favor entropy decrease (cognitive compression). We introduce InstructDiff, a unified framework that operationalizes differential entropy as a domain-adaptive selection criterion through warmup calibration, bi-directional NLL filtering, and entropy-based ranking. Extensive experiments show that InstructDiff achieves 17\% relative improvement over full data training on mathematical reasoning and 52\% for general instruction-following, outperforming prior baselines while using only 10\% of the data.

Problem

• data selection
• domain specificity
• large language models
• supervised fine-tuning
• differential entropy

Innovation

• differential entropy
• domain-adaptive data selection
• instruction tuning
• large language models
• data efficiency