Evaluating an LLM-Powered Chatbot for Cognitive Restructuring: Insights from Mental Health Professionals

2025-01-26

AI Summary

This study investigates the clinical feasibility and ethical risks of large language models (LLMs) in cognitive restructuring (CR)—a core evidence-based psychotherapeutic technique. To this end, we developed a CR-specific chatbot via prompt engineering and conducted an empirical user study involving 19 real participants; their interaction logs were subjected to qualitative analysis and clinical evaluation by licensed mental health professionals—the first study to integrate authentic user engagement with expert clinical assessment. Results indicate that LLMs can execute fundamental CR procedures (e.g., Socratic questioning, empathic reflection, protocol adherence), yet exhibit persistent structural risks: misinterpretation of client cues, boundary violations in therapeutic advice, power imbalances, and excessive positivity bias. Critically, we propose and substantiate a novel “real-time clinical supervision” design paradigm, wherein human experts actively monitor and intervene during AI-assisted sessions. This framework advances both theoretical understanding and practical implementation pathways for safe, clinically integrated AI interventions.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Safety and Robustness
• Philosophy and Ethics of AI: Safety, Robustness & Trustworthiness

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Large language models for search
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Recent advancements in large language models (LLMs) promise to expand mental health interventions by emulating therapeutic techniques, potentially easing barriers to care. Yet there is a lack of real-world empirical evidence evaluating the strengths and limitations of LLM-enabled psychotherapy interventions. In this work, we evaluate an LLM-powered chatbot, designed via prompt engineering to deliver cognitive restructuring (CR), with 19 users. Mental health professionals then examined the resulting conversation logs to uncover potential benefits and pitfalls. Our findings indicate that an LLM-based CR approach has the capability to adhere to core CR protocols, prompt Socratic questioning, and provide empathetic validation. However, issues of power imbalances, advice-giving, misunderstood cues, and excessive positivity reveal deeper challenges, including the potential to erode therapeutic rapport and ethical concerns. We also discuss design implications for leveraging LLMs in psychotherapy and underscore the importance of expert oversight to mitigate these concerns, which are critical steps toward safer, more effective AI-assisted interventions.

Problem

• Large Language Models
• Psychological Therapy
• Ethical Considerations

Innovation

• Large Language Models
• Mental Health Therapy
• Chatbot Design