Perfectly to a Tee: Understanding User Perceptions of Personalized LLM-Enhanced Narrative Interventions

2024-09-25

AI Summary

This study addresses two key challenges in narrative interventions: insufficient individual resonance and difficulty internalizing theory of mind. To this end, we propose a large language model (LLM)-based method for generating personalized narratives tailored to young adults’ specific psychological concerns and cognitive profiles. For the first time, we empirically validate its efficacy via a two-scenario randomized controlled trial (N = 346). Results show that LLM-generated narratives significantly outperform human-written stories in critical information conveyance (p < 0.01), depth of reflection (+23%), and reduction of maladaptive beliefs (Cohen’s d = 0.41), while maintaining equivalent perceived authenticity. We introduce the novel design principle of “credible alignment”—a framework ensuring that LLM-generated narratives simultaneously achieve personalization, clinical effectiveness, and phenomenological authenticity. This work establishes a viable pathway for integrating LLMs into evidence-informed psychological interventions.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Generation
• Humans and AI: Game Design — Procedural Content Generation & Storytelling

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Stories about overcoming personal struggles can effectively illustrate the application of psychological theories in real life, yet they may fail to resonate with individuals' experiences. In this work, we employ large language models (LLMs) to create tailored narratives that acknowledge and address unique challenging thoughts and situations faced by individuals. Our study, involving 346 young adults across two settings, demonstrates that personalized LLM-enhanced stories were perceived to be better than human-written ones in conveying key takeaways, promoting reflection, and reducing belief in negative thoughts. These stories were not only seen as more relatable but also similarly authentic to human-written ones, highlighting the potential of LLMs in helping young adults manage their struggles. The findings of this work provide crucial design considerations for future narrative-based digital mental health interventions, such as the need to maintain relatability without veering into implausibility and refining the wording and tone of AI-enhanced content.

Problem

• Understanding user perceptions of personalized LLM-enhanced narrative interventions
• Creating tailored narratives for unique challenging thoughts and situations
• Evaluating effectiveness of AI-enhanced stories in mental health interventions

Innovation

• Using LLMs to create personalized narratives
• Tailoring stories to individual challenging thoughts
• Enhancing relatability and authenticity with AI