Beyond the Basics: Leveraging Large Language Model for Fine-Grained Medical Entity Recognition

2026-04-18

AI Summary

Fine-grained medical named entity recognition (NER) is crucial for clinical applications, yet existing evaluations of large language models are often confined to coarse-grained categories. This work systematically compares zero-shot, few-shot, and LoRA fine-tuning paradigms within a unified LLaMA3 framework across 18 fine-grained medical entity types. To enhance few-shot learning, the authors propose a novel example selection strategy based on similarity between BioBERT token and sentence embeddings. Experimental results demonstrate that the fine-tuned LLaMA3 achieves an F1 score of 81.24%, substantially outperforming zero-shot (+63.11%) and few-shot (+35.63%) approaches. This study presents the first fair, multi-paradigm comparison under a consistent architecture, advancing the practical deployment of fine-grained medical NER.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Data Mining & Knowledge Management: Linked Open Data, Knowledge Graphs & KB Completion

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Extracting clinically relevant information from unstructured medical narratives such as admission notes, discharge summaries, and emergency case histories remains a challenge in clinical natural language processing (NLP). Medical Entity Recognition (MER) identifies meaningful concepts embedded in these records. Recent advancements in large language models (LLMs) have shown competitive MER performance; however, evaluations often focus on general entity types, offering limited utility for real-world clinical needs requiring finer-grained extraction. To address this gap, we rigorously evaluated the open-source LLaMA3 model for fine-grained medical entity recognition across 18 clinically detailed categories. To optimize performance, we employed three learning paradigms: zero-shot, few-shot, and fine-tuning with Low-Rank Adaptation (LoRA). To further enhance few-shot learning, we introduced two example selection methods based on token- and sentence-level embedding similarity, utilizing a pre-trained BioBERT model. Unlike prior work assessing zero-shot and few-shot performance on proprietary models (e.g., GPT-4) or fine-tuning different architectures, we ensured methodological consistency by applying all strategies to a unified LLaMA3 backbone, enabling fair comparison across learning settings. Our results showed that fine-tuned LLaMA3 surpasses zero-shot and few-shot approaches by 63.11% and 35.63%, respectivel respectively, achieving an F1 score of 81.24% in granular medical entity extraction.

Problem

• Medical Entity Recognition
• Fine-Grained Extraction
• Clinical NLP
• Unstructured Medical Text
• Large Language Models

Innovation

• fine-grained medical entity recognition
• LLaMA3
• LoRA fine-tuning
• few-shot learning
• BioBERT embedding