DreamerNLplus: Interpretable Modeling of Mental Health Dynamics from Social Media Timelines using Hybrid Rule-Based and RAG Methods

2026-05-21

AI Summary

This work addresses the challenge of modeling mental health dynamics in social media timelines by proposing an interpretable framework that integrates a rule-based engine with Retrieval-Augmented Generation (RAG). The approach combines LLM-based data augmentation, DeBERTa for psychological state classification, random forest temporal regression, and a few-shot prompting strategy using Llama 3.1 to effectively capture evolving user mental states. Evaluated on the CLPsych 2026 shared task, the method achieved second place in the official ranking for the timeline summarization task. Notably, the RAG module demonstrated superior performance in detecting psychological improvement (ranked 1st) and ranked third for deterioration cases, thereby validating both the effectiveness and interpretability of the proposed framework in modeling mental health dynamics.

Abstract

We present DreamerNLplus, a hybrid framework for modeling mental health dynamics from social media timelines in the CLPsych 2026 shared task. Our system addresses three tasks: psychological state modeling, temporal change detection, and sequence-level summarization. For Task 1, we combine LLM-based data augmentation, DeBERTa classification, and Random Forest regression for structured state prediction. For Task 2, we use few-shot prompting with a locally deployed Llama 3.1 model to detect Switch and Escalation events using short-term temporal context. For Task 3.1, we explore both a deterministic rule-based summarization pipeline and a few-shot LLM-based approach, ranking \textbf{2nd} officially. Our RAG-based method achieves strong performance in Task 3.2, ranking \textbf{1st} for Improvement and \textbf{3rd} for Deterioration, demonstrating its ability to capture recurrent psychological change patterns across timelines. Our analysis reveals key challenges, including the mismatch between classification and regression performance, the difficulty of modeling temporal transitions, and the disagreement between semantic and similarity-based evaluation metrics. These findings highlight the complexity of modeling mental health dynamics and motivate future work on unified evaluation frameworks. We share our code and prompts at https://github.com/4dpicture/CLPsych2026

Problem

• mental health dynamics
• social media timelines
• psychological state modeling
• temporal change detection
• sequence-level summarization

Innovation

• hybrid framework
• retrieval-augmented generation (RAG)
• mental health dynamics
• few-shot prompting
• temporal change detection