Overview of the PsyDefDetect Shared Task at BioNLP 2026: Detecting Levels of Psychological Defense Mechanisms in Supportive Conversations

2026-05-24

AI Summary

This study addresses the automatic identification of psychological defense mechanisms in emotional support conversations, framing the task as a context-aware classification problem across nine clinically validated defense categories based on the Defense Mechanism Rating Scales (DMRS) framework. As the first work to introduce fine-grained psychological defense mechanisms into natural language processing, it presents PsyDefConv, a high-quality annotated dataset, and proposes a classification approach that integrates large language models with theory-driven strategies. Experimental results demonstrate that the best-performing system achieves a macro F1-score of 0.420, significantly outperforming baseline methods. Nevertheless, the system still faces challenges such as over-prediction of high-frequency classes and class imbalance, highlighting the complexity of modeling nuanced psychological constructs in conversational contexts.

Abstract

We present an overview of PsyDefDetect, the shared task on detecting levels of psychological defense mechanisms in emotional support dialogues, co-located with BioNLP@ACL 2026. Grounded in the clinically validated Defense Mechanism Rating Scales (DMRS) framework, the task asks systems to classify a target seeker utterance, given its preceding dialogue context, into one of nine categories: seven hierarchical DMRS levels plus two auxiliary labels. Participants worked on PsyDefConv, a newly released corpus of 200 dialogues and 2336 help-seeker utterances annotated under DMRS with substantial inter-annotator agreement. The task attracted 172 participants on CodaBench who produced 563 submissions, with 21 teams officially registering their results for the final ranking. The best system achieved a macro F1-score of 0.420, surpassing the strongest fine-tuned baseline reported in the dataset paper by a notable margin, yet leaving clear headroom. Our analysis highlights (i) a persistent tendency to over-predict the majority High-Adaptive class, (ii) a widening gap between accuracy and macro-F1 that reveals class-imbalance sensitivity, and (iii) the value of theory-aware and LLM-based approaches for fine-grained defensive-function classification. We release all task materials and invite the community to continue work on this novel intersection of clinical psychology and NLP.

Problem

• psychological defense mechanisms
• emotional support dialogues
• Defense Mechanism Rating Scales
• utterance classification
• clinical psychology

Innovation

• psychological defense mechanisms
• Defense Mechanism Rating Scales (DMRS)
• emotional support dialogues
• fine-grained classification
• LLM-based approaches