Expresso-AI: Explainable Video-Based Deep Learning Models for Depression Diagnosis

2026-06-24

AI Summary

This study addresses the limited emotion specificity and poor interpretability that hinder the clinical applicability of existing automated depression diagnosis methods. To overcome these challenges, the authors propose an interpretable deep learning model based on facial videos, which—uniquely—employs a deep convolutional neural network (DCNN) pretrained for action recognition to assess depression severity. The approach integrates spatiotemporal facial expression semantics, saliency analysis of facial regions, and temporal dynamics modeling. Experiments on the AVEC depression dataset demonstrate that the method outperforms current single-frame facial diagnosis benchmarks and significantly enhances model transparency, predictive performance, and clinical credibility through dual visual and quantitative explanations.

Abstract

Given the widespread prevalence of depression and its consequential impact on individuals and society, it is crucial to obtain objective measures for early diagnosis and intervention. As a multidisciplinary topic, these objective measures should be interpretable and accessible to health care professionals, ensuring effective collaboration and treatment planning in the realm of mental health care. Even though current automated depression diagnosis approaches improved over the last decade, a critical gap exists as they often lack affect-specificity and interpretability, limiting their practical application and potential impact on mental health care. In particular, interpretability from temporal activities from videos when deep models are used is not fully explored. In this study, we present a novel framework for analyzing Deep Neural Networks' decisions when trained on facial videos, specifically focusing on automatic depression severity diagnosis. By fine-tuning Deep Convolutional Neural Networks (DCNN) pre-trained on Action Recognition datasets on depression severity facial videos from AVEC depression dataset, our framework is able to interpret the model's saliency maps by examining face regions and temporal expression semantics. Our approach generates both visual and quantitative explanations for the model's decisions, providing greater insight into its reasoning. In addition to this interpretability, our video-based modeling has improved upon previous single-face benchmarks for visual depression diagnosis, resulting in enhanced predictive performance. Overall, our work demonstrates the successful development of a framework capable of generating hypotheses from a facial model's decisions while simultaneously improving depression's predictive capabilities.

Problem

• depression diagnosis
• explainable AI
• video-based deep learning
• interpretability
• temporal activities

Innovation

• explainable AI
• video-based depression diagnosis
• temporal expression semantics
• saliency maps
• deep convolutional neural networks