GIFT: Guided Fine-Tuning and Transfer for Enhancing Instruction-Tuned Language Models

2026-05-02

AI Summary

This work proposes GIFT, a novel framework that actively leverages the confidence signals from instruction-tuned models to guide low-rank adapter training during task-specific fine-tuning—addressing the limitation of existing approaches that treat instruction models merely as passive targets for merging. By incorporating confidence-guided fine-tuning, low-rank adaptation, and model merging, GIFT effectively balances task specialization with general instruction-following capabilities. The optimized adapters are merged back into the original instruction model, preserving its broad competence while enhancing performance on specific tasks. Evaluated across multiple mathematical and knowledge-intensive benchmarks, GIFT significantly outperforms standard fine-tuning and prevailing transfer learning methods, demonstrating superior generalization and test-time scalability.

Abstract

A promising paradigm for adapting instruction-tuned language models is to learn task-specific updates on a pretrained base model and subsequently merge them into the instruction-tuned model. However, existing approaches typically treat the instruction-tuned model as a passive target that is only involved at the final merging stage, without guiding the training process. We propose GIFT (Guided Fine-Tuning and Transfer), a simple and efficient framework that incorporates guidance from the instruction model into task adaptation. GIFT fine-tunes a low-rank adapter on the pretrained base model using confidence signals derived from the instruction-tuned model. The learned adapter is then merged into the instruction-tuned model, yielding task-specialized models that preserve general instruction-following behavior. We evaluate GIFT on mathematical and knowledge-intensive benchmarks across multiple model families and scales. Results show that GIFT consistently outperforms direct fine-tuning and representative transfer-based baselines, while maintaining robust generalization and favorable test-time scaling behavior.

Problem

• instruction-tuned language models
• task-specific adaptation
• model transfer
• fine-tuning

Innovation

• guided fine-tuning
• instruction-tuned language models
• low-rank adaptation
• model merging
• transfer learning