Deep literature reviews: an application of fine-tuned language models to migration research

2025-04-17

AI Summary

This study addresses key limitations of conventional bibliometric methods in systematic reviews of human migration research—namely, low efficiency, insufficient analytical depth, and delayed trend detection. Methodologically, we develop a hybrid framework integrating bibliometrics with domain-adapted large language models (LLMs): leveraging over 20,000 migration-related publications, we fine-tune an open-source LLM via domain adaptation and implement an error-focused human-in-the-loop validation protocol to precisely calibrate the model as a migration research “expert.” Contributions include: (1) the first LLM-driven, systematic identification of emerging research trends and critical knowledge gaps; (2) empirical evidence of pronounced structural biases in climate-migration literature—particularly the marginalization of health-related drivers such as air/water pollution and infectious diseases; and (3) substantial improvements in review breadth, analytical depth, and cross-study consistency.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Search and Optimization: Metareasoning and Metaheuristics
• Application Domains: Humanities & Computational Social Science

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

This paper presents a hybrid framework for literature reviews that augments traditional bibliometric methods with large language models (LLMs). By fine-tuning open-source LLMs, our approach enables scalable extraction of qualitative insights from large volumes of research content, enhancing both the breadth and depth of knowledge synthesis. To improve annotation efficiency and consistency, we introduce an error-focused validation process in which LLMs generate initial labels and human reviewers correct misclassifications. Applying this framework to over 20000 scientific articles about human migration, we demonstrate that a domain-adapted LLM can serve as a"specialist"model - capable of accurately selecting relevant studies, detecting emerging trends, and identifying critical research gaps. Notably, the LLM-assisted review reveals a growing scholarly interest in climate-induced migration. However, existing literature disproportionately centers on a narrow set of environmental hazards (e.g., floods, droughts, sea-level rise, and land degradation), while overlooking others that more directly affect human health and well-being, such as air and water pollution or infectious diseases. This imbalance highlights the need for more comprehensive research that goes beyond physical environmental changes to examine their ecological and societal consequences, particularly in shaping migration as an adaptive response. Overall, our proposed framework demonstrates the potential of fine-tuned LLMs to conduct more efficient, consistent, and insightful literature reviews across disciplines, ultimately accelerating knowledge synthesis and scientific discovery.

Problem

• Enhancing literature reviews with fine-tuned LLMs for migration research
• Improving annotation efficiency via error-focused validation in LLM-assisted reviews
• Identifying research gaps in climate-induced migration literature

Innovation

• Hybrid framework combining bibliometrics and LLMs
• Fine-tuned LLMs for scalable qualitative extraction
• Error-focused validation with human-AI collaboration