Beyond Similarity: A Gradient-based Graph Method for Instruction Tuning Data Selection

2025-02-16

AI Summary

To address the scarcity and suboptimal selection of instruction-tuning data in domain adaptation, this paper proposes Gradient-based Instruction Graph Modeling and Gradient Walk—a novel data selection framework. Unlike conventional methods relying on single-instruction similarity, our approach explicitly models the joint distribution and dependency among instructions by incorporating gradient similarity into an instruction graph structure, enabling relational awareness and dynamic selection. Technically, it integrates hybrid gradient graph construction, instruction-level gradient similarity measurement, and graph neural network–assisted ranking. Evaluated across multiple low-resource domains, the method achieves an average 12.7% improvement in downstream task performance, accelerates training convergence by 38%, and significantly enhances generalization capability.

Technology Category

• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Search and Optimization: Learning to Search
• Constraint Satisfaction and Optimization: Constraint Learning and Acquisition

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Large language models (LLMs) have shown great potential across various industries due to their remarkable ability to generalize through instruction tuning. However, the limited availability of domain-specific data significantly hampers their performance on specialized tasks. While existing methods primarily focus on selecting training data from general datasets that are similar to the target domain, they often fail to consider the joint distribution of instructions, resulting in inefficient learning and suboptimal knowledge transfer. To address these challenges, we introduce G2IS (Gradient-based Graph Instruction Selection), a novel method that constructs a mixed gradient-based instruction graph to capture the joint distribution and interdependencies between instructions. By accounting for the relationships between instructions, G2IS improves domain adaptation efficiency. Additionally, we propose a gradient walk algorithm to refine the data selection process, enhancing both training effectiveness and efficiency. Our experiments demonstrate that G2IS outperforms traditional methods across various domain adaptation tasks, yielding significant performance gains, particularly in complex, data-scarce scenarios. These results underscore the potential of G2IS in advancing the development of large, domain-specific models.

Problem

• Enhance domain-specific LLMs performance
• Optimize instruction tuning data selection
• Improve domain adaptation efficiency

Innovation

• Gradient-based Graph Instruction Selection
• Captures joint instruction distribution
• Enhances domain adaptation efficiency