Natural Language Processing Psychometrics

2026-08-07

AI Summary

Current NLP models for mental health prediction often lack explicit definitions of psychological constructs—such as affect, syntax, and semantics—and exhibit limited generalizability. This study reframes psychological language prediction as a psychometric problem by integrating psychometric principles with interpretable AI. Specifically, we employ large language models with controlled personality settings to generate questionnaire responses accompanied by explanations, combining emotional features, textual-form mental networks, and demographic variables. Using random forests and SHAP analysis, we quantify the contribution of linguistic features to distinct psychological constructs. The model explains up to 70.8% of variance in life satisfaction, 55.7% in depression, and 76.0% in anxiety on synthetic data. Notably, emotional and network features alone achieve 68% accuracy in distinguishing clinical from control groups in real transcripts and demonstrate robust cross-text-type prediction without retraining (r = 0.91), highlighting both the potential and limitations of synthetic data for uncovering biases and simulating clinical reasoning.

Abstract

Natural Language Processing (NLP) models predicting mental health outcomes rarely specify what they measure: contextual knowledge, emotional content, or syntactic structure. NLP Psychometrics treats psychological prediction from text as a psychometric problem, linking scores to interpretable linguistic evidence and testing beyond the training text format. Nine LLMs, conditioned on controlled personas (cognitive digital shadows), completed psychometric questionnaires with textual explanations per item. We extracted emotional profiles and syntactic-semantic structure via textual forma mentis networks, combined with personality and sociodemographic variables in ablated random forest (RF) regressors, using SHAP to identify which features drove performance and in which direction. Full RF models explained up to 70.8% of variance in life satisfaction (SWLS), 55.7% in depression (PHQ-9), and, for DASS-21, 68.5% depression, 76.0% anxiety, 72.4% stress. Sociodemographics alone explained no meaningful variance in depression, anxiety, or stress, but did so for life satisfaction, where emotion features and income were the strongest predictors; neuroticism and network topology instead dominated depression and anxiety, reversing direction between them. Without retraining, RF models separated diaries from low- and high-score personas ($r$ up to 0.91) and, using only network/emotion features, classified clinical from control participants in real transcripts with up to 68% accuracy. These results show the promise and limits of synthetic data: LLM personas can expose model biases, recover patterns consistent with clinical rumination, and support psychometric prediction from human text without a matched questionnaire, but cannot substitute for human validation. NLP Psychometrics makes these distinctions explicit, measurable, and testable through interpretable AI and network/emotional features.

Problem

• Natural Language Processing
• Psychometrics
• Mental Health Prediction
• Interpretable AI
• Linguistic Evidence

Innovation

• NLP Psychometrics
• textual forma mentis networks
• interpretable AI
• cognitive digital shadows
• psychometric prediction