Large Language Models in Qualitative Research: Uses, Tensions, and Intentions

2024-10-09

AI Summary

This study examines the methodological appropriateness, ethical legitimacy, and hermeneutic compatibility of large language models (LLMs) in qualitative research, particularly regarding researcher values and interpretive traditions. Drawing on in-depth interviews with 20 qualitative researchers and employing thematic analysis informed by human-centered AI design principles, the study systematically identifies— for the first time—six core ethical and methodological tensions arising from LLM integration across qualitative research phases. Its primary contribution is a novel “researcher-intention-centered” ethical practice framework and corresponding AI tool design principles, addressing critical theoretical and operational gaps in LLM governance within qualitative methodology. Key outputs include an actionable LLM usage recommendation checklist and an interpretive-hermeneutic research support paradigm. Collectively, these advances establish a methodological foundation for the responsible, reflexive integration of generative AI into qualitative inquiry.

Technology Category

• Philosophy and Ethics of AI: Morality & Value-based AI
• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Ethics — Bias, Fairness, Transparency & Privacy

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work

Abstract

Qualitative researchers use tools to collect, sort, and analyze their data. Should qualitative researchers use large language models (LLMs) as part of their practice? LLMs could augment qualitative research, but it is unclear if their use is appropriate, ethical, or aligned with qualitative researchers' goals and values. We interviewed twenty qualitative researchers to investigate these tensions. Many participants see LLMs as promising interlocutors with attractive use cases across the stages of research, but wrestle with their performance and appropriateness. Participants surface concerns regarding the use of LLMs while protecting participant interests, and call attention to an urgent lack of norms and tooling to guide the ethical use of LLMs in research. We document the rapid and broad adoption of LLMs across surfaces, which can interfere with intentional use vital to qualitative research. We use the tensions surfaced by our participants to outline recommendations for researchers considering using LLMs in qualitative research and design principles for LLM-assisted qualitative research tools.

Problem

• Assessing appropriateness of LLMs in qualitative research
• Ethical concerns in LLM use for participant protection
• Lack of norms for ethical LLM use in research

Innovation

• LLMs augment qualitative research stages
• Ethical concerns in LLM use addressed
• Design principles for LLM-assisted tools