A Complete Survey on Contemporary Methods, Emerging Paradigms and Hybrid Approaches for Few-Shot Learning

2024-02-05

AI Summary

Few-shot learning (FSL) addresses the generalization bottleneck of deep learning under data-scarce conditions. This work conducts a systematic literature review and paradigmatic reconstruction to unify FSL frameworks: it is the first to incorporate in-context learning into a coherent FSL taxonomy; introduces novel meta-learning subcategories—including neural processes and probabilistic meta-learning—to extend classical meta-learning theory; and establishes the first comprehensive FSL taxonomy encompassing supervised, semi-supervised, and unsupervised settings. Through cross-paradigm comparative analysis and application-domain mapping, we construct the most extensive FSL knowledge graph to date, clarifying core challenges—such as task distribution shift and prior modeling bias—and identifying key future directions, including enhanced interpretability and cross-modal transfer. The resulting framework provides both theoretical foundations and practical guidelines for algorithmic innovation and cross-domain deployment.

Technology Category

• Machine Learning: Multimodal Learning
• Search and Optimization: Learning to Search
• Multiagent Systems: Multiagent Learning

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods, algorithms and applications for the development of semantic models, knowledge graphs and other forms of structured data models with machine-interpretable semantics

Abstract

Despite the widespread success of deep learning, its intense requirements for vast amounts of data and extensive training make it impractical for various real-world applications where data is scarce. In recent years, Few-Shot Learning (FSL) has emerged as a learning paradigm that aims to address these limitations by leveraging prior knowledge to enable rapid adaptation to novel learning tasks. Due to its properties that highly complement deep learning's data-intensive needs, FSL has seen significant growth in the past few years. This survey provides a comprehensive overview of both well-established methods as well as recent advancements in the FSL field. The presented taxonomy extends previously proposed ones by incorporating emerging FSL paradigms, such as in-context learning, along with novel categories within the meta-learning paradigm for FSL, including neural processes and probabilistic meta-learning. Furthermore, a holistic overview of FSL is provided by discussing hybrid FSL approaches that extend FSL beyond the typically examined supervised learning setting. The survey also explores FSL's diverse applications across various domains. Finally, recent trends shaping the field, outstanding challenges, and promising future research directions are discussed.

Problem

• Limited Data
• Deep Learning
• Training Efficiency

Innovation

• Few-Shot Learning
• Meta-Learning
• Data-Efficient Deep Learning