Detecting HIV-Related Stigma in Clinical Narratives Using Large Language Models

2026-04-08

AI Summary

This study addresses the absence of automated tools for identifying HIV-related stigma in clinical texts, which hinders the assessment of its psychosocial impacts. We present the first fine-grained, open-source annotated dataset tailored to clinical settings, encompassing four dimensions of HIV stigma: concerns about public attitudes, disclosure worries, negative self-image, and personalized stigma. Data augmentation was performed using keyword filtering combined with clinical word embeddings. We systematically evaluated multiple large language models under zero-shot and few-shot settings, finding that GatorTron-large achieved the best performance (Micro F1 = 0.62), while 5-shot prompting substantially improved generative models (LLaMA-8B reached 0.59). Negative self-image was the most readily identifiable dimension, whereas personalized stigma posed the greatest challenge. Additionally, we developed the first NLP tool specifically designed for HIV stigma detection in clinical applications.

Abstract

Human immunodeficiency virus (HIV)-related stigma is a critical psychosocial determinant of health for people living with HIV (PLWH), influencing mental health, engagement in care, and treatment outcomes. Although stigma-related experiences are documented in clinical narratives, there is a lack of off-the-shelf tools to extract and categorize them. This study aims to develop a large language model (LLM)-based tool for identifying HIV stigma from clinical notes. We identified clinical notes from PLWH receiving care at the University of Florida (UF) Health between 2012 and 2022. Candidate sentences were identified using expert-curated stigma-related keywords and iteratively expanded via clinical word embeddings. A total of 1,332 sentences were manually annotated across four stigma subscales: Concern with Public Attitudes, Disclosure Concerns, Negative Self-Image, and Personalized Stigma. We compared GatorTron-large and BERT as encoder-based baselines, and GPT-OSS-20B, LLaMA-8B, and MedGemma-27B as generative LLMs, under zero-shot and few-shot prompting. GatorTron-large achieved the best overall performance (Micro F1 = 0.62). Few-shot prompting substantially improved generative model performance, with 5-shot GPT-OSS-20B and LLaMA-8B achieving Micro-F1 scores of 0.57 and 0.59, respectively. Performance varied by stigma subscale, with Negative Self-Image showing the highest predictability and Personalized Stigma remaining the most challenging. Zero-shot generative inference exhibited non-trivial failure rates (up to 32%). This study develops the first practical NLP tool for identifying HIV stigma in clinical notes.

Problem

• HIV-related stigma
• clinical narratives
• natural language processing
• stigma detection
• large language models

Innovation

• HIV-related stigma
• large language models
• clinical narratives
• few-shot prompting
• natural language processing