🤖 AI Summary
This work addresses the challenges of integrating high-dimensional, continuous, and longitudinal wearable health data with text-centric large language models (LLMs), where diverse user intents and heterogeneous sensor modalities hinder fixed inference pipelines. To overcome these limitations, we propose the first query-adaptive agent framework tailored for wearable health question answering. The framework leverages an LLM to dynamically orchestrate reasoning pathways, flexibly invoking specialized analytical tools, pretrained models, and external knowledge bases to achieve deep fusion of multimodal sensor data and language understanding. Evaluated on four public datasets spanning three major health domains, our system improves accuracy by 24% over strong baselines. In blind assessments by 12 medical experts and 8 end users, the generated responses demonstrated significantly higher practical utility and clinical plausibility compared to existing approaches.
📝 Abstract
Language models are remarkably capable at medical question answering, in some cases surpassing the accuracy of general physicians. However, answering questions about wearable health data remains challenging and understudied, as these ubiquitous sensors produce continuous, high-dimensional, and longitudinal data, which is non-trivial to align with text-centric distributions in LLM pretraining. The diversity of sensor modalities and user intents cannot be effectively handled by a fixed reasoning workflow or a single pretrained foundation model. To address these challenges, we propose WEQA, a query-adaptive agent framework that unifies LLM reasoning with specialized wearable analytical and modeling tools. An LLM controller is employed to synthesize execution plans and dynamically route each query to the appropriate combination of sensor analysis and pretrained models, and perform grounded response auditing with external knowledge. We also curate a benchmark spanning four open wearable datasets comprising analytic and predictive tasks in three different health domains. Experiments show that our framework is 24% more accurate than LLM and agentic baselines, and a blinded study with 12 medical experts and 8 users shows substantial gains in usefulness and clinical soundness.