A Layered Multi-Expert Framework for Long-Context Mental Health Assessments

2025-01-20

AI Summary

To address hallucination, inconsistent reasoning, and declining reliability of large language models (LLMs) in long-text mental health assessment, this paper proposes Stacked Multi-Model Reasoning (SMMR), a hierarchical multi-expert framework. SMMR introduces a novel “short-context subtask isolation–long-context integration decoupling” mechanism, orchestrating multiple LLMs and lightweight domain-specific models to hierarchically decompose assessment tasks. Cross-layer output integration and a PHQ-8-aligned multi-scale fusion strategy enhance decision consistency. A diversified “second-opinion” mechanism further improves clinical nuance recognition and high-risk judgment reliability. Evaluated on the DAIC-WOZ dataset and 48 real-world clinical cases, SMMR significantly outperforms single-model baselines in accuracy and F1-score, reducing PHQ-8 scoring error by 23.6%.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Large language models for search
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Long-form mental health assessments pose unique challenges for large language models (LLMs), which often exhibit hallucinations or inconsistent reasoning when handling extended, domain-specific contexts. We introduce Stacked Multi-Model Reasoning (SMMR), a layered framework that leverages multiple LLMs and specialized smaller models as coequal 'experts'. Early layers isolate short, discrete subtasks, while later layers integrate and refine these partial outputs through more advanced long-context models. We evaluate SMMR on the DAIC-WOZ depression-screening dataset and 48 curated case studies with psychiatric diagnoses, demonstrating consistent improvements over single-model baselines in terms of accuracy, F1-score, and PHQ-8 error reduction. By harnessing diverse 'second opinions', SMMR mitigates hallucinations, captures subtle clinical nuances, and enhances reliability in high-stakes mental health assessments. Our findings underscore the value of multi-expert frameworks for more trustworthy AI-driven screening.

Problem

• Long-term Assessment
• Mental Health Evaluation
• Model Reliability

Innovation

• Stacked Multi-Model Reasoning (SMMR)
• Hierarchical Expert System
• Mental Health Assessment