When LLM Therapists Become Salespeople: Evaluating Large Language Models for Ethical Motivational Interviewing

2025-03-30

AI Summary

This study addresses a critical ethical gap in large language models’ (LLMs) application to motivational interviewing (MI): despite possessing moderate-to-high MI knowledge, LLMs frequently generate or overlook unethical practices—particularly manipulative dialogue strategies. To tackle this, we introduce the first systematic MI ethics evaluation framework and propose Chain-of-Ethic prompting, a novel technique that elicits ethically grounded responses through iterative, contrastive reasoning. We conduct empirical evaluations across leading closed- and open-source LLMs using multi-round comparative experiments and ethics sensitivity testing. Results demonstrate significant improvements: +32.7% in accuracy of ethically appropriate response generation and +41.5% in detection of unethical practices. Our work delivers a verifiable, deployable technical solution for safe AI-delivered psychological services, backed by rigorous empirical evidence and grounded in MI-specific ethical principles.

Technology Category

• Natural Language Processing: Ethics — Bias, Fairness, Transparency & Privacy
• Machine Learning: Large Multimodal Models (LMMs)
• Philosophy and Ethics of AI: Safety, Robustness & Trustworthiness

Application Category

• User Modeling, Personalization and Recommendation: Accountability, Transparency, and Ethics for personalization
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Large language models (LLMs) have been actively applied in the mental health field. Recent research shows the promise of LLMs in applying psychotherapy, especially motivational interviewing (MI). However, there is a lack of studies investigating how language models understand MI ethics. Given the risks that malicious actors can use language models to apply MI for unethical purposes, it is important to evaluate their capability of differentiating ethical and unethical MI practices. Thus, this study investigates the ethical awareness of LLMs in MI with multiple experiments. Our findings show that LLMs have a moderate to strong level of knowledge in MI. However, their ethical standards are not aligned with the MI spirit, as they generated unethical responses and performed poorly in detecting unethical responses. We proposed a Chain-of-Ethic prompt to mitigate those risks and improve safety. Finally, our proposed strategy effectively improved ethical MI response generation and detection performance. These findings highlight the need for safety evaluations and guidelines for building ethical LLM-powered psychotherapy.

Problem

• Evaluating LLMs' ethical awareness in motivational interviewing
• Assessing risks of unethical MI practices by LLMs
• Improving ethical response generation and detection in LLMs

Innovation

• Evaluating LLM ethical awareness in MI
• Proposing Chain-of-Ethic prompt strategy
• Improving ethical response generation and detection