A Multi-Agent Audit Framework for High-Stakes Reasoning: Evaluation and Interpretability in Clinical Mental Health Screening

2026-06-19

AI Summary

This study addresses the limitations of single large language models in zero-shot, high-stakes clinical mental health screening—specifically their susceptibility to hallucination and insufficient interpretability—by proposing the first multi-agent auditing framework tailored for psychological assessment. The framework orchestrates four synergistic stages: perception, retrieval-augmented generation, chain-of-thought clinical reasoning, and audit verification, thereby establishing a traceable and verifiable AI-assisted decision pipeline. A cross-agent validation mechanism is incorporated to mitigate reasoning drift. Implemented via a modular LangChain workflow with locally deployed open-source large language models, the system reduces the mean absolute error in PHQ-8 depression score prediction on the DAIC-WOZ dataset from 5.35 to 5.02, significantly outperforming single-agent baselines while delivering highly interpretable diagnostic rationales.

Abstract

High-stakes reasoning tasks necessitate transparent and verifiable workflows, yet conventional single-model large language models (LLMs) often struggle with hallucination and low interpretability under zero-shot paradigms. To address this general AI challenge, we propose a Multi-Agent Audit Framework that simulates a collaborative, multi-step verification process. We empirically validate this architecture in the sensitive domain of clinical mental health screening using a modular LangChain workflow. Our framework decomposes the reasoning process into a Perception Agent, Knowledge Retrieval-Augmented Generation (RAG), Chain-of-Thought (CoT) clinical inference, and a critical Audit verification stage. We evaluated this framework on the DAIC-WOZ dataset using locally deployed open-source models. Experimental results demonstrate that our multi-agent pipeline significantly outperforms single-agent baselines, reducing the Mean Absolute Error (MAE) for PHQ-8 depression severity prediction from 5.35 to 5.02. By exposing cross-agent validation traces, the framework mitigates reasoning drift and provides highly interpretable diagnostic rationales, offering a generalizable paradigm for reliable AI-assisted decision support beyond isolated model scaling. We make data and code open access on GitHub for replicability.

Problem

• high-stakes reasoning
• hallucination
• interpretability
• clinical mental health screening
• large language models

Innovation

• Multi-Agent Audit Framework
• Retrieval-Augmented Generation (RAG)
• Chain-of-Thought Reasoning
• Interpretable AI
• Clinical Mental Health Screening