Framing Responsible Design of AI Mental Well-Being Support: AI as Primary Care, Nutritional Supplement, or Yoga Instructor?

2026-02-02

AI Summary

This study addresses the responsible design of non-clinical large language model (LLM) tools for mental health support by proposing a multidimensional analogy framework. Through semi-structured expert interviews and policy document analysis, the authors conceptualize AI tools as analogous to pharmaceuticals, dietary supplements, yoga instructors, or primary care providers—each analogy clarifying expected benefits, mechanisms of active ingredient delivery, risk profiles, and boundaries of responsibility. The framework operates independently of specific algorithms, instead offering a conceptual foundation to define design objectives and ethical obligations for AI-enabled mental health tools. It provides developers, designers, and users with actionable guidance for evaluating such systems and allocating responsibilities, thereby advancing the normative and accountable deployment of responsible AI in mental health contexts.

Technology Category

• Cognitive Modeling & Cognitive Systems: Analogy
• Philosophy and Ethics of AI: Safety, Robustness & Trustworthiness
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Millions of people now use non-clinical Large Language Model (LLM) tools like ChatGPT for mental well-being support. This paper investigates what it means to design such tools responsibly, and how to operationalize that responsibility in their design and evaluation. By interviewing experts and analyzing related regulations, we found that designing an LLM tool responsibly involves: (1) Articulating the specific benefits it guarantees and for whom. Does it guarantee specific, proven relief, like an over-the-counter drug, or offer minimal guarantees, like a nutritional supplement? (2) Specifying the LLM tool's"active ingredients"for improving well-being and whether it guarantees their effective delivery (like a primary care provider) or not (like a yoga instructor). These specifications outline an LLM tool's pertinent risks, appropriate evaluation metrics, and the respective responsibilities of LLM developers, tool designers, and users. These analogies - LLM tools as supplements, drugs, yoga instructors, and primary care providers - can scaffold further conversations about their responsible design.

Problem

• responsible AI
• mental well-being
• Large Language Models
• AI design
• non-clinical support

Innovation

• responsible AI design
• large language models
• mental well-being support
• design analogies
• AI accountability