Fine-tuning LLMs for Passive Depression Severity Estimation from AI Mental Health Dialogue

2026-06-16

AI Summary

This study addresses the limitations of traditional depression monitoring, which relies on self-report scales like the PHQ-9 and suffers from low user compliance, leading to missing data and bias. To overcome this, the authors propose a passive assessment method that requires no active user input, leveraging only conversational text between users and an AI mental health application. By fine-tuning the Qwen3.5-27B large language model, the approach directly predicts total PHQ-9 scores for continuous depression severity monitoring. The key innovation lies in achieving high-accuracy, end-to-end estimation across the full spectrum of depressive severity without any additional clinical data, using pseudo-labels generated by Claude Opus and an iterative training strategy. Evaluated on a test set of 842 users, the model achieves a mean absolute error (MAE) of 2.6, root mean square error (RMSE) of 4.0, Pearson correlation of 0.80, and an AUC of 0.91 for detecting PHQ-9 ≥ 10, with AUC exceeding 0.87 across all clinical severity thresholds.

Abstract

Depression is the leading cause of disability worldwide, and early detection of symptom change is essential for timely intervention. Validated instruments such as the Patient Health Questionnaire-9 (PHQ-9) support symptom monitoring at scale, but real-world completion rates are low, introducing response bias and systematic missingness. Passive approaches that infer severity from routinely generated data could close this gap. We address this by predicting PHQ-9 total scores directly from transcripts of conversations between users and an AI mental health application, requiring only conversation text and no additional clinical data. We fine-tune a Qwen3.5-27B backbone with a regression head, augment 3,111 ground-truth labels with pseudolabels generated by a reasoning model (Claude Opus) and iteratively trained intermediate models, for a combined dataset of 6,283 users. On a held-out test set of 842 users, our best model achieves MAE = 2.6, RMSE = 4.0, Pearson r = 0.80, and AUC = 0.91 at the PHQ-9 >= 10 clinical threshold. We also find AUC > 0.87 at every severity threshold from PHQ-9 >= 3 to PHQ-9 >= 24, demonstrating that the model captures depression severity across the full clinical spectrum. This work opens the door to passive, continuous symptom monitoring in AI mental health platforms, without requiring users to complete self-report measures.

Problem

• passive depression severity estimation
• AI mental health dialogue
• PHQ-9 prediction
• symptom monitoring
• response bias

Innovation

• passive depression estimation
• LLM fine-tuning
• pseudolabel augmentation
• PHQ-9 prediction
• AI mental health dialogue