Exploring the Potential of Human-LLM Synergy in Advancing Qualitative Analysis: A Case Study on Mental-Illness Stigma

📅 2024-05-09
🏛️ arXiv.org
📈 Citations: 2
Influential: 0
📄 PDF
🤖 AI Summary
This study addresses two key challenges in qualitative analysis: theoretical rigidity and the unidirectional application of large language models (LLMs) as mere coding aids. To overcome these, we propose a novel human–AI co-constructive paradigm for theory generation. Focusing on mental illness stigma, we introduce CHALET—a methodology integrating manual coding, prompt engineering, disagreement-driven iterative negotiation, and cross-dimensional thematic modeling—to enable bidirectional feedback and collaborative inductive coding. Unlike conventional LLM-assisted approaches that treat AI as a static coding tool, CHALET facilitates autonomous emergence of novel theoretical insights. Empirical evaluation demonstrates that human–AI collaboration significantly enhances theoretical sensitivity and conceptual originality, successfully uncovering latent stigma themes across cognitive, affective, and behavioral dimensions. This work establishes a reproducible, scalable methodological framework for theory-driven qualitative research, advancing the integration of AI not as an auxiliary instrument but as a co-theorist in interpretive inquiry.

Technology Category

Application Category

📝 Abstract
Qualitative analysis is a challenging, yet crucial aspect of advancing research in the field of Human-Computer Interaction (HCI). Recent studies show that large language models (LLMs) can perform qualitative coding within existing schemes, but their potential for collaborative human-LLM discovery and new insight generation in qualitative analysis is still underexplored. To bridge this gap and advance qualitative analysis by harnessing the power of LLMs, we propose CHALET, a novel methodology that leverages the human-LLM collaboration paradigm to facilitate conceptualization and empower qualitative research. The CHALET approach involves LLM-supported data collection, performing both human and LLM deductive coding to identify disagreements, and performing collaborative inductive coding on these disagreement cases to derive new conceptual insights. We validated the effectiveness of CHALET through its application to the attribution model of mental-illness stigma, uncovering implicit stigmatization themes on cognitive, emotional and behavioral dimensions. We discuss the implications for future research, methodology, and the transdisciplinary opportunities CHALET presents for the HCI community and beyond.
Problem

Research questions and friction points this paper is trying to address.

Enhancing theory-driven qualitative analysis through human-LLM collaboration
Exploring human-LLM synergy to generate new insights beyond initial theory
Developing a method for iterative coding and conceptualization of qualitative data
Innovation

Methods, ideas, or system contributions that make the work stand out.

Human-LLM iterative coding and disagreement analysis
CHALET approach for collaborative theory-driven qualitative analysis
Uncovering latent themes across cognitive, emotional, behavioral dimensions
🔎 Similar Papers
No similar papers found.