EHSAN: Leveraging ChatGPT in a Hybrid Framework for Arabic Aspect-Based Sentiment Analysis in Healthcare

2025-08-04

AI Summary

Arabic medical sentiment analysis has long suffered from dialectal diversity and a severe scarcity of aspect-level annotations. To address this, we introduce the first explainable Arabic medical domain aspect-based sentiment analysis (ABSA) dataset. Our method innovatively combines large language model (LLM)-generated pseudo-labels—produced via ChatGPT—with rigorous human verification, providing explicit, step-by-step reasoning for each annotation. The dataset is released in three supervision variants—low-, medium-, and high-resource—to empirically demonstrate that high performance can be sustained with minimal human annotation effort. Leveraging this resource, we fine-tune Arabic-specific Transformer models to jointly perform aspect term extraction and sentiment classification. Experiments show only marginal performance degradation under fully automated labeling, while aspect category pruning significantly improves accuracy. Our approach thus establishes a robust, transparent, and scalable framework for low-resource ABSA in specialized domains.

Technology Category

• Natural Language Processing: Sentiment Analysis, Stylistic Analysis, and Argument Mining
• Machine Learning: Large Multimodal Models (LMMs)
• Knowledge Representation and Reasoning: Diagnosis and Abductive Reasoning

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling

Abstract

Arabic-language patient feedback remains under-analysed because dialect diversity and scarce aspect-level sentiment labels hinder automated assessment. To address this gap, we introduce EHSAN, a data-centric hybrid pipeline that merges ChatGPT pseudo-labelling with targeted human review to build the first explainable Arabic aspect-based sentiment dataset for healthcare. Each sentence is annotated with an aspect and sentiment label (positive, negative, or neutral), forming a pioneering Arabic dataset aligned with healthcare themes, with ChatGPT-generated rationales provided for each label to enhance transparency. To evaluate the impact of annotation quality on model performance, we created three versions of the training data: a fully supervised set with all labels reviewed by humans, a semi-supervised set with 50% human review, and an unsupervised set with only machine-generated labels. We fine-tuned two transformer models on these datasets for both aspect and sentiment classification. Experimental results show that our Arabic-specific model achieved high accuracy even with minimal human supervision, reflecting only a minor performance drop when using ChatGPT-only labels. Reducing the number of aspect classes notably improved classification metrics across the board. These findings demonstrate an effective, scalable approach to Arabic aspect-based sentiment analysis (SA) in healthcare, combining large language model annotation with human expertise to produce a robust and explainable dataset. Future directions include generalisation across hospitals, prompt refinement, and interpretable data-driven modelling.

Problem

• Under-analyzed Arabic patient feedback due to dialect diversity
• Lack of aspect-level sentiment labels for automated assessment
• Need for explainable Arabic healthcare sentiment dataset

Innovation

• Hybrid pipeline combining ChatGPT and human review
• First explainable Arabic healthcare sentiment dataset
• Fine-tuned transformers for aspect and sentiment classification