Large Language Models for Large-Scale, Rigorous Qualitative Analysis in Applied Health Services Research

2025-11-02

Research Square

AI Summary

This study addresses the persistent challenge in large-scale, multicenter health services research of balancing efficiency and methodological rigor in qualitative analysis. It proposes a generalizable, model- and task-agnostic human-AI collaboration framework that systematically integrates large language models (LLMs) into real-world qualitative workflows, supporting diverse analytical objectives such as thematic synthesis and deductive coding. Applied to a federated study of diabetes care across qualified health centers, the framework efficiently coded 167 interview transcripts and generated practitioner-oriented comparative feedback reports that timely informed intervention refinement. By maintaining analytic rigor while substantially accelerating the qualitative process, this work establishes a scalable and transferable paradigm for leveraging LLMs to enhance qualitative research in complex health services contexts.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Knowledge Representation and Reasoning: Qualitative Reasoning

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• Search and Retrieval-Augmented AI: Large language models for search

Abstract

Large language models (LLMs) show promise for improving the efficiency of qualitative analysis in large, multi-site health-services research. Yet methodological guidance for LLM integration into qualitative analysis and evidence of their impact on real-world research methods and outcomes remain limited. We developed a model- and task-agnostic framework for designing human-LLM qualitative analysis methods to support diverse analytic aims. Within a multi-site study of diabetes care at Federally Qualified Health Centers (FQHCs), we leveraged the framework to implement human-LLM methods for (1) qualitative synthesis of researcher-generated summaries to produce comparative feedback reports and (2) deductive coding of 167 interview transcripts to refine a practice-transformation intervention. LLM assistance enabled timely feedback to practitioners and the incorporation of large-scale qualitative data to inform theory and practice changes. This work demonstrates how LLMs can be integrated into applied health-services research to enhance efficiency while preserving rigor, offering guidance for continued innovation with LLMs in qualitative research.

Problem

• large language models
• qualitative analysis
• health services research
• methodological guidance
• rigor

Innovation

• large language models
• qualitative analysis
• health services research
• human-AI collaboration
• deductive coding