🤖 AI Summary
This study addresses the challenges of clinical information extraction from low-resource language medical texts, including data scarcity, high annotation costs, and privacy constraints. The authors propose a lightweight, tuning-free two-stage approach: first, anonymized Persian medical transcripts are translated into English using Aya-expanse-8B; then, thirteen binary clinical attributes are extracted in parallel via few-shot prompting on compact open-source models such as Qwen2.5, Llama-3.1/3.2, and Gemma-3. Experimental results show that Qwen2.5-7B-Instruct achieves the best performance (median macro F1 of 0.899, MCC of 0.797). Larger models significantly outperform smaller ones in sensitivity and MCC. Translating to English improves sensitivity at a slight cost to specificity. Extraction of physiological symptoms proves relatively reliable, whereas psychological and complex signs remain challenging. This work demonstrates the effectiveness of jointly optimizing model scale and input language, offering a practical, privacy-preserving pathway for low-resource clinical settings.
📝 Abstract
Extracting clinical information from medical transcripts in low-resource languages remains a significant challenge in healthcare natural language processing (NLP). This study evaluates a two-step pipeline combining Aya-expanse-8B as a Persian-to-English translation model with five open-source small language models (SLMs) -- Qwen2.5-7B-Instruct, Llama-3.1-8B-Instruct, Llama-3.2-3B-Instruct, Qwen2.5-1.5B-Instruct, and Gemma-3-1B-it -- for binary extraction of 13 clinical features from 1,221 anonymized Persian transcripts collected at a cancer palliative care call center. Using a few-shot prompting strategy without fine-tuning, models were assessed on macro-averaged F1-score, Matthews Correlation Coefficient (MCC), sensitivity, and specificity to account for class imbalance. Qwen2.5-7B-Instruct achieved the highest overall performance (median macro-F1: 0.899; MCC: 0.797), while Gemma-3-1B-it showed the weakest results. Larger models (7B--8B parameters) consistently outperformed smaller counterparts in sensitivity and MCC. A bilingual analysis of Aya-expanse-8B revealed that translating Persian transcripts to English improved sensitivity, reduced missing outputs, and boosted metrics robust to class imbalance, though at the cost of slightly lower specificity and precision. Feature-level results showed reliable extraction of physiological symptoms across most models, whereas psychological complaints, administrative requests, and complex somatic features remained challenging. These findings establish a practical, privacy-preserving blueprint for deploying open-source SLMs in multilingual clinical NLP settings with limited infrastructure and annotation resources, and highlight the importance of jointly optimizing model scale and input language strategy for sensitive healthcare applications.