Selected work

Representative Papers

FHIR-AgentBench: Benchmarking LLM Agents for Realistic Interoperable EHR Question Answering

Sep 12, 2025arXiv.org

Existing benchmarks inadequately assess large language model (LLM) agents’ capabilities on real-world, HL7 FHIR-standardized clinical data. Method: We introduce FHIR-AgentBench—the first realistic clinical question-answering benchmark grounded in FHIR, comprising 2,931 clinically grounded questions derived from authentic electronic health record (EHR) scenarios. For the first time, we systematically evaluate LLM agents within the FHIR resource model framework, comparing strategies including API-based vs. domain-specific tool usage, single-turn vs. multi-turn interaction, and natural-language vs. code-based generation—thereby uncovering core challenges: retrieval ambiguity, cross-resource relational modeling, and multi-step clinical reasoning. Contribution/Results: We publicly release an open-source dataset and evaluation suite. Empirical analysis identifies critical performance bottlenecks of current agent approaches on structured clinical queries, advancing reproducible, interoperable research on clinical AI agents.

1 citationsRead paper

The complexities of patient-centred conversational artificial intelligence

Jul 09, 2026

Current health chatbots often rely on idealized patient simulations, struggling to accommodate the diversity in real users’ expression, emotional states, and communication strategies, which can lead to triage bias. Addressing this limitation, this study leverages 2,053 real clinical dialogues to propose the first multidimensional patient simulator that explicitly models clinical content, emotional state, dialogue strategy, and communication style. These dimensions are systematically integrated into a large language model (LLM)-based triage evaluation framework. Experimental results show that dialogues generated by the simulator are nearly indistinguishable from real conversations in human evaluations (identification accuracy of only 55%). Moreover, varying communication styles significantly affect the urgency assessment accuracy of four leading LLMs, highlighting the critical role of communicative diversity in ensuring fairness in AI-driven triage.

0 citationsRead paper

Detecting Clinical Discrepancies in Health Coaching Agents: A Dual-Stream Memory and Reconciliation Architecture

Apr 29, 2026

This work addresses the clinical safety risks posed by conventional memory architectures in persistent health coaching agents, which overwrite prior information when integrating patient self-reports—vulnerable to recall bias—with structured but often outdated electronic health records (EHRs). To mitigate this, the authors propose a dual-stream memory architecture that strictly segregates unstructured patient narratives from standardized FHIR-based EHR data. A dedicated coordination engine systematically compares extracted memories, categorizing clinical discrepancies by type, severity, and FHIR resource. This approach achieves the first explicit separation and systematic reconciliation of patient-reported and EHR-derived information, revealing that 13.6% of error cascades originate during memory extraction and that loss of clinical detail primarily stems from inadequate information extraction from unstructured dialogue. Evaluated over 675 longitudinal sessions, the coordination engine attained an 84.4% detection rate for predefined clinical discrepancies and an 86.7% recall for safety-critical differences, demonstrating the feasibility and necessity of clinically grounded memory validation.

0 citationsRead paper

STEER: Inference-Time Risk Control via Constrained Quality-Diversity Search

Feb 02, 2026

This work addresses the rigidity of large language models (LLMs) in ordinal decision-making tasks—such as clinical triage—that require balancing sensitivity and specificity, noting their lack of tunable risk-control mechanisms. To overcome this limitation, the authors propose STEER, a framework that constructs a diverse ensemble of natural language personas via offline constrained quality-diversity search and dynamically selects a persona at inference time using a single interpretable parameter corresponding to a user-specified risk percentile. This enables monotonic control over decision conservativeness without additional training. Evaluated on two clinical triage benchmarks, STEER achieves a broader behavioral spectrum than temperature sampling or static role ensembles, maintaining high accuracy on critical cases while offering enhanced controllability for ambiguous decisions.

0 citationsRead paper

Integrating Genomics into Multimodal EHR Foundation Models

Oct 24, 2025

Traditional EHR models struggle to integrate clinical data with genetic risk—such as polygenic risk scores (PRS)—hindering accurate disease prediction and personalized care. To address this, we propose the first multimodal EHR foundation model that natively embeds PRS as an independent modality, trained on the All of Us cohort to enable deep genomic–clinical coupling. Methodologically, we extend multimodal deep learning by integrating generative AI and transfer learning, supporting interpretable analysis and fine-grained risk stratification. Experiments demonstrate significant performance gains over baselines in predicting key conditions—including type 2 diabetes (AUC improvement ≥0.04)—and highlight potential for real-world evidence generation and personalized health management. Our core contribution lies in the native multimodal representation of PRS and joint representation learning with EHR data, establishing a new paradigm for genetically informed clinical modeling.

0 citationsRead paper

Recent publications

Latest Papers

The complexities of patient-centred conversational artificial intelligence

Jul 09, 2026

Current health chatbots often rely on idealized patient simulations, struggling to accommodate the diversity in real users’ expression, emotional states, and communication strategies, which can lead to triage bias. Addressing this limitation, this study leverages 2,053 real clinical dialogues to propose the first multidimensional patient simulator that explicitly models clinical content, emotional state, dialogue strategy, and communication style. These dimensions are systematically integrated into a large language model (LLM)-based triage evaluation framework. Experimental results show that dialogues generated by the simulator are nearly indistinguishable from real conversations in human evaluations (identification accuracy of only 55%). Moreover, varying communication styles significantly affect the urgency assessment accuracy of four leading LLMs, highlighting the critical role of communicative diversity in ensuring fairness in AI-driven triage.

0 citationsRead paper

Detecting Clinical Discrepancies in Health Coaching Agents: A Dual-Stream Memory and Reconciliation Architecture

Apr 29, 2026

This work addresses the clinical safety risks posed by conventional memory architectures in persistent health coaching agents, which overwrite prior information when integrating patient self-reports—vulnerable to recall bias—with structured but often outdated electronic health records (EHRs). To mitigate this, the authors propose a dual-stream memory architecture that strictly segregates unstructured patient narratives from standardized FHIR-based EHR data. A dedicated coordination engine systematically compares extracted memories, categorizing clinical discrepancies by type, severity, and FHIR resource. This approach achieves the first explicit separation and systematic reconciliation of patient-reported and EHR-derived information, revealing that 13.6% of error cascades originate during memory extraction and that loss of clinical detail primarily stems from inadequate information extraction from unstructured dialogue. Evaluated over 675 longitudinal sessions, the coordination engine attained an 84.4% detection rate for predefined clinical discrepancies and an 86.7% recall for safety-critical differences, demonstrating the feasibility and necessity of clinically grounded memory validation.

0 citationsRead paper

STEER: Inference-Time Risk Control via Constrained Quality-Diversity Search

Feb 02, 2026

This work addresses the rigidity of large language models (LLMs) in ordinal decision-making tasks—such as clinical triage—that require balancing sensitivity and specificity, noting their lack of tunable risk-control mechanisms. To overcome this limitation, the authors propose STEER, a framework that constructs a diverse ensemble of natural language personas via offline constrained quality-diversity search and dynamically selects a persona at inference time using a single interpretable parameter corresponding to a user-specified risk percentile. This enables monotonic control over decision conservativeness without additional training. Evaluated on two clinical triage benchmarks, STEER achieves a broader behavioral spectrum than temperature sampling or static role ensembles, maintaining high accuracy on critical cases while offering enhanced controllability for ambiguous decisions.

0 citationsRead paper

Integrating Genomics into Multimodal EHR Foundation Models

Oct 24, 2025

Traditional EHR models struggle to integrate clinical data with genetic risk—such as polygenic risk scores (PRS)—hindering accurate disease prediction and personalized care. To address this, we propose the first multimodal EHR foundation model that natively embeds PRS as an independent modality, trained on the All of Us cohort to enable deep genomic–clinical coupling. Methodologically, we extend multimodal deep learning by integrating generative AI and transfer learning, supporting interpretable analysis and fine-grained risk stratification. Experiments demonstrate significant performance gains over baselines in predicting key conditions—including type 2 diabetes (AUC improvement ≥0.04)—and highlight potential for real-world evidence generation and personalized health management. Our core contribution lies in the native multimodal representation of PRS and joint representation learning with EHR data, establishing a new paradigm for genetically informed clinical modeling.

0 citationsRead paper

An AI-Based Behavioral Health Safety Filter and Dataset for Identifying Mental Health Crises in Text-Based Conversations

Oct 13, 2025

Large language models (LLMs) frequently generate harmful responses in mental health crisis conversations, necessitating highly sensitive safety filtering mechanisms. To address this, we propose the Verily Behavioral Health Safety Filter (VBHSF), a machine learning–based classifier trained on clinically annotated crisis data from the Verily and NVIDIA Aegis datasets. VBHSF prioritizes minimizing false negatives—thereby maximizing sensitivity—while maintaining strong generalization and robustness across diverse psychological crisis scenarios. Experimental evaluation demonstrates state-of-the-art performance: on the Verily dataset, VBHSF achieves 0.990 sensitivity and 0.992 specificity (F1 = 0.939); on the NVIDIA dataset, it attains 0.982 sensitivity and 0.921 accuracy—both significantly surpassing OpenAI’s Omni Moderation Latest and NVIDIA’s NeMo Guardrails. To our knowledge, VBHSF is the first open-source safety filter specifically designed for mental health emergencies that simultaneously delivers high sensitivity and robust real-world reliability.

0 citationsRead paper