Standardization of Psychiatric Diagnoses -- Role of Fine-tuned LLM Consortium and OpenAI-gpt-oss Reasoning LLM Enabled Decision Support System

2025-10-29

AI Summary

Psychiatric diagnosis relies heavily on subjective clinical interviews, resulting in poor inter-rater and patient–clinician diagnostic consistency and limited reliability. To address this, we propose the first collaborative diagnostic architecture integrating a fine-tuned LLM ensemble with a reasoning-oriented LLM (OpenAI GPT-4o). Our approach introduces an LLM-agent coordination mechanism and a multi-model consensus decision algorithm, trained and deployed on real-world clinician–patient dialogue data using three domain-adapted LLMs. The architecture ensures interpretability and transparency while enhancing diagnostic standardization. Experimental evaluation demonstrates high accuracy and robustness in mental health assessment across diverse clinical scenarios. We have developed a deployable prototype platform and are actively collaborating with the U.S. Army Medical Research Team to advance clinical eHealth implementation.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Cognitive Modeling & Cognitive Systems: Agent Architectures
• Knowledge Representation and Reasoning: Diagnosis and Abductive Reasoning

Application Category

• Economics, Online Markets and Human Computation: Architectures and workflows that use LLMs for crowd work
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

The diagnosis of most mental disorders, including psychiatric evaluations, primarily depends on dialogues between psychiatrists and patients. This subjective process can lead to variability in diagnoses across clinicians and patients, resulting in inconsistencies and challenges in achieving reliable outcomes. To address these issues and standardize psychiatric diagnoses, we propose a Fine-Tuned Large Language Model (LLM) Consortium and OpenAI-gpt-oss Reasoning LLM-enabled Decision Support System for the clinical diagnosis of mental disorders. Our approach leverages fine-tuned LLMs trained on conversational datasets involving psychiatrist-patient interactions focused on mental health conditions (e.g., depression). The diagnostic predictions from individual models are aggregated through a consensus-based decision-making process, refined by the OpenAI-gpt-oss reasoning LLM. We propose a novel method for deploying LLM agents that orchestrate communication between the LLM consortium and the reasoning LLM, ensuring transparency, reliability, and responsible AI across the entire diagnostic workflow. Experimental results demonstrate the transformative potential of combining fine-tuned LLMs with a reasoning model to create a robust and highly accurate diagnostic system for mental health assessment. A prototype of the proposed platform, integrating three fine-tuned LLMs with the OpenAI-gpt-oss reasoning LLM, was developed in collaboration with the U.S. Army Medical Research Team in Norfolk, Virginia, USA. To the best of our knowledge, this work represents the first application of a fine-tuned LLM consortium integrated with a reasoning LLM for clinical mental health diagnosis paving the way for next-generation AI-powered eHealth systems aimed at standardizing psychiatric diagnoses.

Problem

• Standardizing subjective psychiatric diagnoses to reduce clinician variability
• Addressing inconsistencies in mental disorder assessments through AI systems
• Improving reliability of psychiatric evaluations using LLM consensus methods

Innovation

• Fine-tuned LLMs analyze psychiatrist-patient dialogue data
• Consensus decision-making aggregates predictions from multiple models
• OpenAI reasoning LLM refines diagnostic workflow for reliability