Transforming Wearable Data into Health Insights using Large Language Model Agents

2024-06-10

arXiv.org

AI Summary

To address the challenge of extracting personalized health insights from wearable-device data—tasks requiring complex numerical reasoning that general-purpose large language models (LLMs) inadequately support—this paper proposes Personal Health Insights Agent (PHIA), the first systematic LLM-agent framework for personal behavioral health data analysis. PHIA integrates tool invocation, code generation, external knowledge retrieval, and multi-turn collaborative reasoning. We construct a dual-dimension benchmark comprising 4,000+ questions—covering numeric factual queries and open-ended health questions. Experiments show PHIA achieves ≥84% accuracy on numeric factual questions and ≥83% on open-ended health questions. Rigorous evaluation over 650 person-hours by domain experts confirms its practical utility and reliability. Key contributions include: (1) introducing the PHIA agent paradigm; (2) releasing the first wearable-health-analysis-specific benchmark; and (3) realizing a tool-augmented multimodal reasoning architecture.

Technology Category

• Humans and AI: Human-Aware Planning and Behavior Prediction
• Cognitive Modeling & Cognitive Systems: Agent Architectures
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Agentic search

Abstract

Despite the proliferation of wearable health trackers and the importance of sleep and exercise to health, deriving actionable personalized insights from wearable data remains a challenge because doing so requires non-trivial open-ended analysis of these data. The recent rise of large language model (LLM) agents, which can use tools to reason about and interact with the world, presents a promising opportunity to enable such personalized analysis at scale. Yet, the application of LLM agents in analyzing personal health is still largely untapped. In this paper, we introduce the Personal Health Insights Agent (PHIA), an agent system that leverages state-of-the-art code generation and information retrieval tools to analyze and interpret behavioral health data from wearables. We curate two benchmark question-answering datasets of over 4000 health insights questions. Based on 650 hours of human and expert evaluation we find that PHIA can accurately address over 84% of factual numerical questions and more than 83% of crowd-sourced open-ended questions. This work has implications for advancing behavioral health across the population, potentially enabling individuals to interpret their own wearable data, and paving the way for a new era of accessible, personalized wellness regimens that are informed by data-driven insights.

Problem

• Deriving personalized insights from wearable tracker data
• Overcoming complex numerical reasoning challenges with LLMs
• Enabling accessible data-driven personal health analysis

Innovation

• LLM agents with code generation
• Multistep reasoning with information retrieval
• Personal Health Insights Agent system