TAMA: A Human-AI Collaborative Thematic Analysis Framework Using Multi-Agent LLMs for Clinical Interviews

2025-03-26

AI Summary

Traditional thematic analysis (TA) for clinical interview data is labor-intensive, while existing large language model (LLM)-based TA methods lack medical domain expertise and interpretability. Method: We propose a human-in-the-loop, multi-agent LLM framework tailored for pediatric cardiology. It integrates domain-specific cardiac knowledge via role-based prompt engineering, structured agent dialogue, iterative consensus negotiation, and closed-loop human feedback to enable expert-guided theme generation and validation. Results: Evaluated on parent interviews from children with anomalous aortic origin of a coronary artery (AAOCA), our approach significantly improves theme recall, coverage, and discriminability over prior LLM-TA methods, while reducing manual effort by over 60%. This work introduces the first explainable, editable, and verifiable multi-agent TA paradigm for medicine, establishing a robust, efficient human–AI collaboration pathway for clinical qualitative research.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Multiagent Systems: Modeling other Agents
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Agentic search
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Thematic analysis (TA) is a widely used qualitative approach for uncovering latent meanings in unstructured text data. TA provides valuable insights in healthcare but is resource-intensive. Large Language Models (LLMs) have been introduced to perform TA, yet their applications in healthcare remain unexplored. Here, we propose TAMA: A Human-AI Collaborative Thematic Analysis framework using Multi-Agent LLMs for clinical interviews. We leverage the scalability and coherence of multi-agent systems through structured conversations between agents and coordinate the expertise of cardiac experts in TA. Using interview transcripts from parents of children with Anomalous Aortic Origin of a Coronary Artery (AAOCA), a rare congenital heart disease, we demonstrate that TAMA outperforms existing LLM-assisted TA approaches, achieving higher thematic hit rate, coverage, and distinctiveness. TAMA demonstrates strong potential for automated TA in clinical settings by leveraging multi-agent LLM systems with human-in-the-loop integration by enhancing quality while significantly reducing manual workload.

Problem

• Automating thematic analysis in clinical interviews using multi-agent LLMs
• Reducing manual workload in healthcare thematic analysis
• Improving thematic hit rate, coverage, and distinctiveness in TA

Innovation

• Multi-agent LLMs for thematic analysis
• Human-AI collaboration in clinical interviews
• Structured agent conversations enhance coherence