OnRL-RAG: Real-Time Personalized Mental Health Dialogue System

2025-04-02

AI Summary

To address the limitations of large language models (LLMs) in dynamically adapting to and delivering individualized interventions within real-time mental health dialogues, this paper proposes OnRL-RAG—an online reinforcement learning–driven retrieval-augmented generation framework. OnRL-RAG integrates RAG, online RL, human feedback alignment (RLHF), and multimodal reasoning (leveraging GPT-4o and Gemini-1.5), trained on authentic psychological survey data from 2,028 university students across stress, anxiety, and depression scenarios. It establishes the first continuous feedback loop for RAG and enables population-specific response optimization—overcoming the temporal rigidity of static RAG and the lack of personalization in generic fine-tuning. Experimental results demonstrate statistically significant improvements over standard RAG and baseline LLMs across core psychotherapeutic dialogue metrics: response relevance, empathic accuracy, and individualized adaptation. OnRL-RAG thus provides a deployable technical paradigm for daily AI-powered mental health support.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Language Grounding & Multi-modal NLP
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Large language models (LLMs) have been widely used for various tasks and applications. However, LLMs and fine-tuning are limited to the pre-trained data. For example, ChatGPT's world knowledge until 2021 can be outdated or inaccurate. To enhance the capabilities of LLMs, Retrieval-Augmented Generation (RAG), is proposed to augment LLMs with additional, new, latest details and information to LLMs. While RAG offers the correct information, it may not best present it, especially to different population groups with personalizations. Reinforcement Learning from Human Feedback (RLHF) adapts to user needs by aligning model responses with human preference through feedback loops. In real-life applications, such as mental health problems, a dynamic and feedback-based model would continuously adapt to new information and offer personalized assistance due to complex factors fluctuating in a daily environment. Thus, we propose an Online Reinforcement Learning-based Retrieval-Augmented Generation (OnRL-RAG) system to detect and personalize the responding systems to mental health problems, such as stress, anxiety, and depression. We use an open-source dataset collected from 2028 College Students with 28 survey questions for each student to demonstrate the performance of our proposed system with the existing systems. Our system achieves superior performance compared to standard RAG and simple LLM via GPT-4o, GPT-4o-mini, Gemini-1.5, and GPT-3.5. This work would open up the possibilities of real-life applications of LLMs for personalized services in the everyday environment. The results will also help researchers in the fields of sociology, psychology, and neuroscience to align their theories more closely with the actual human daily environment.

Problem

• Enhancing LLMs with real-time, personalized mental health responses
• Adapting RAG systems to user needs via reinforcement learning
• Improving dialogue systems for stress, anxiety, and depression

Innovation

• Online Reinforcement Learning-based RAG system
• Personalized mental health response adaptation
• Dynamic feedback for real-time user alignment