The Rise of Small Language Models in Healthcare: A Comprehensive Survey

2025-04-23

AI Summary

Large language models (LLMs) in healthcare face critical challenges of patient data privacy risks and computational resource constraints, hindering clinical deployment. Method: This paper introduces the first three-dimensional taxonomy for small language models (SLMs) tailored to clinical adoption—spanning NLP task types, stakeholder roles, and stages of the care continuum—and proposes a novel tripartite methodology: (1) architectural lightweighting, (2) clinical adaptation via prompt engineering, instruction tuning, and inference enhancement, and (3) sustainable deployment through model compression and inference optimization. Contribution/Results: Empirical multi-task evaluation demonstrates that SLMs achieve performance parity with LLMs on core clinical tasks—including diagnostic support and clinical note understanding—while delivering superior efficiency–performance trade-offs. The work releases an open-source, structured repository of models, tools, and protocols, establishing a reproducible methodological foundation and practical implementation guide for health informatics in privacy-sensitive and resource-constrained environments.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Large language models for search
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Despite substantial progress in healthcare applications driven by large language models (LLMs), growing concerns around data privacy, and limited resources; the small language models (SLMs) offer a scalable and clinically viable solution for efficient performance in resource-constrained environments for next-generation healthcare informatics. Our comprehensive survey presents a taxonomic framework to identify and categorize them for healthcare professionals and informaticians. The timeline of healthcare SLM contributions establishes a foundational framework for analyzing models across three dimensions: NLP tasks, stakeholder roles, and the continuum of care. We present a taxonomic framework to identify the architectural foundations for building models from scratch; adapting SLMs to clinical precision through prompting, instruction fine-tuning, and reasoning; and accessibility and sustainability through compression techniques. Our primary objective is to offer a comprehensive survey for healthcare professionals, introducing recent innovations in model optimization and equipping them with curated resources to support future research and development in the field. Aiming to showcase the groundbreaking advancements in SLMs for healthcare, we present a comprehensive compilation of experimental results across widely studied NLP tasks in healthcare to highlight the transformative potential of SLMs in healthcare. The updated repository is available at Github

Problem

• Addressing data privacy and resource constraints with small language models in healthcare.
• Providing a taxonomic framework for healthcare SLMs across NLP tasks and care continuum.
• Showcasing SLM advancements through optimized models and experimental results in healthcare.

Innovation

• SLMs offer scalable healthcare solutions
• Taxonomic framework categorizes healthcare SLMs
• Compression techniques enhance SLM accessibility