🤖 AI Summary
To address critical shortages of psychiatric personnel and prolonged patient wait times in low-resource settings, this study proposes an automated clinical summarization method for Mental Status Examination (MSE) texts. We systematically evaluate and fine-tune open-weight large language models—including LLaMA-2, Falcon, Qwen, and Phi-3—on a newly curated mental health screening dataset, employing supervised fine-tuning (SFT) and a dual evaluation framework combining ROUGE metrics with expert human assessment. Our approach significantly enhances clinical relevance and cross-dataset generalizability: it achieves ROUGE-1 = 0.810 and ROUGE-L = 0.764 on the proprietary dataset, while maintaining robust performance on the public D4 benchmark. The core contribution is the first MSE-specific fine-tuning paradigm, explicitly designed to generate clinically actionable summaries. This framework delivers a scalable, high-fidelity decision-support tool for mental health screening in resource-constrained regions.
📝 Abstract
Improving mental health support in developing countries is a pressing need. One potential solution is the development of scalable, automated systems to conduct diagnostic screenings, which could help alleviate the burden on mental health professionals. In this work, we evaluate several state-of-the-art Large Language Models (LLMs), with and without fine-tuning, on our custom dataset for generating concise summaries from mental state examinations. We rigorously evaluate four different models for summary generation using established ROUGE metrics and input from human evaluators. The results highlight that our top-performing fine-tuned model outperforms existing models, achieving ROUGE-1 and ROUGE-L values of 0.810 and 0.764, respectively. Furthermore, we assessed the fine-tuned model's generalizability on a publicly available D4 dataset, and the outcomes were promising, indicating its potential applicability beyond our custom dataset.