Understanding the Generalization of In-Context Learning in Transformers: An Empirical Study

2025-03-19

AI Summary

This work addresses the poorly understood generalization boundaries and pronounced cross-task fragility of large language models (e.g., GPT-4, LLaMA-3) in in-context learning (ICL). We propose the first task-centric, three-dimensional ICL generalization framework—spanning cross-problem, intra-problem, and intra-task levels—and conduct systematic empirical evaluation across diverse tasks including function fitting, API calling, and translation. Our findings reveal that Transformers exhibit strong intra-task and intra-problem generalization but severely lack cross-problem generalization. Crucially, increasing task diversity in training data significantly improves ICL generalization to unseen tasks; task-mixed training yields up to a 37% absolute accuracy gain. These results provide actionable, theory-grounded guidance for ICL data curation and model pretraining, along with a unified evaluation protocol for rigorous benchmarking of ICL generalization capabilities.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Computer Vision: Large Vision Models

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Large language models for search
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Large language models (LLMs) like GPT-4 and LLaMA-3 utilize the powerful in-context learning (ICL) capability of Transformer architecture to learn on the fly from limited examples. While ICL underpins many LLM applications, its full potential remains hindered by a limited understanding of its generalization boundaries and vulnerabilities. We present a systematic investigation of transformers' generalization capability with ICL relative to training data coverage by defining a task-centric framework along three dimensions: inter-problem, intra-problem, and intra-task generalization. Through extensive simulation and real-world experiments, encompassing tasks such as function fitting, API calling, and translation, we find that transformers lack inter-problem generalization with ICL, but excel in intra-task and intra-problem generalization. When the training data includes a greater variety of mixed tasks, it significantly enhances the generalization ability of ICL on unseen tasks and even on known simple tasks. This guides us in designing training data to maximize the diversity of tasks covered and to combine different tasks whenever possible, rather than solely focusing on the target task for testing.

Problem

• Investigates generalization boundaries of in-context learning in transformers.
• Explores inter-problem, intra-problem, and intra-task generalization capabilities.
• Identifies training data diversity as key to enhancing ICL generalization.

Innovation

• Systematic investigation of ICL generalization boundaries
• Task-centric framework for three generalization dimensions
• Enhanced ICL generalization with diverse training data