In-context Learning vs. Instruction Tuning: The Case of Small and Multilingual Language Models

2025-03-03

AI Summary

This work investigates performance disparities in instruction-following capability between in-context learning (ICL) and instruction tuning for small-scale multilingual language models (English/French/Spanish), and presents the first systematic evaluation of direct preference optimization (DPO) for aligning base models without reinforcement learning from human feedback (RLHF). We conduct controlled cross-lingual experiments under low-resource and small-model constraints, comparing three paradigms: ICL, supervised instruction tuning, and DPO. Results show that ICL degrades significantly on small models and non-English languages; instruction tuning exhibits superior robustness and stability; while DPO partially mitigates ICL’s performance drop, its compensatory effect remains limited. The core contribution lies in empirically revealing the language- and scale-sensitivity of ICL’s generalization boundary, and in establishing—through rigorous experimentation—the alignment efficacy limits of DPO in lightweight multilingual settings.

Technology Category

• Natural Language Processing: (Large) Language Models
• Machine Learning: Large Multimodal Models (LMMs)
• Search and Optimization: Learning to Search

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Instruction following is a critical ability for Large Language Models to perform downstream tasks. The standard approach to instruction alignment has relied on a specific phase of model tuning over curated instruction datasets, optionally complemented with an alignment step over human preferences. Recent work has shown the potential of in-context learning (ICL) alternatives to guide base models towards instruction following. This type of approach is particularly relevant to extend instruction following across languages and models of varying sizes adapted to different types of usage. In this work we compare ICL and instruction fine-tuning in English, French and Spanish, on Small Language Models, and provide experimental results on applying Direct Preference Optimisation (DPO) over base models. Our results show that scenarios involving multilingual and smaller models result in downgraded ICL instruction following performance, only partially mitigated by DPO alignment. This study aims to further our understanding of current strengths and limitations of alternative methods for instruction following.

Problem

• Compare in-context learning and instruction tuning for multilingual models.
• Evaluate instruction following in small language models across languages.
• Assess Direct Preference Optimisation impact on multilingual instruction alignment.

Innovation

• Compares in-context learning with instruction tuning
• Uses Direct Preference Optimisation on base models
• Tests multilingual and small language models