Hearing Health in Home Healthcare: Leveraging LLMs for Illness Scoring and ALMs for Vocal Biomarker Extraction
🤖 AI Summary
Remote health monitoring in home settings lacks interpretable, multimodal AI methods that integrate unstructured clinical speech with physiological data.
Method: We propose an automated patient health assessment framework for home healthcare that (1) employs a large language model (LLM) to fuse unstructured SOAP-style voice transcripts with structured vital signs—enabling generation of interpretable, composite disease scores; and (2) introduces a multi-stage audio preprocessing pipeline leveraging an audio-language model (ALM) to extract vocal biomarkers from real-world, unconstrained home recordings and produce natural-language explanations.
Results: Experiments show strong agreement between LLM-derived scores and clinical outcomes (Pearson *r* > 0.85); notably, SOAP transcripts demonstrate superior predictive power over physiological metrics alone. The ALM successfully identifies health-relevant acoustic patterns in non-clinical home environments—a first—and generates human-readable interpretations. This work establishes a deployable, interpretable, multimodal AI paradigm for remote health monitoring.
Application Category
📝 Abstract
The growing demand for home healthcare calls for tools that can support care delivery. In this study, we explore automatic health assessment from voice using real-world home care visit data, leveraging the diverse patient information it contains. First, we utilize Large Language Models (LLMs) to integrate Subjective, Objective, Assessment, and Plan (SOAP) notes derived from unstructured audio transcripts and structured vital signs into a holistic illness score that reflects a patient's overall health. This compact representation facilitates cross-visit health status comparisons and downstream analysis. Next, we design a multi-stage preprocessing pipeline to extract short speech segments from target speakers in home care recordings for acoustic analysis. We then employ an Audio Language Model (ALM) to produce plain-language descriptions of vocal biomarkers and examine their association with individuals' health status. Our experimental results benchmark both commercial and open-source LLMs in estimating illness scores, demonstrating their alignment with actual clinical outcomes, and revealing that SOAP notes are substantially more informative than vital signs. Building on the illness scores, we provide the first evidence that ALMs can identify health-related acoustic patterns from home care recordings and present them in a human-readable form. Together, these findings highlight the potential of LLMs and ALMs to harness heterogeneous in-home visit data for better patient monitoring and care.
Problem
Research questions and friction points this paper is trying to address.
Automating health assessment from voice data in home care
Integrating SOAP notes and vital signs into illness scores
Extracting vocal biomarkers through acoustic analysis of recordings
Innovation
Methods, ideas, or system contributions that make the work stand out.
LLMs integrate SOAP notes into illness scores
Multi-stage pipeline extracts speech for acoustic analysis
ALMs describe vocal biomarkers in plain language
🔎 Similar Papers
No similar papers found.
