HypEHR: Hyperbolic Modeling of Electronic Health Records for Efficient Question Answering

2026-04-22

AI Summary

This work addresses the high deployment cost of existing large language models and their neglect of clinical data hierarchies in electronic health record (EHR) question answering by proposing HypEHR—a compact architecture grounded in Lorentzian hyperbolic geometry. HypEHR is the first to introduce hyperbolic space into EHR-QA, embedding medical codes, visit records, and questions while leveraging the ICD ontology hierarchy through pretraining and hierarchy-aware regularization. The model incorporates a geometrically consistent cross-attention mechanism and type-specific pointer heads to better capture structural relationships. Evaluated on two EHR-QA benchmarks constructed from MIMIC-IV, HypEHR achieves performance comparable to much larger models with significantly fewer parameters, effectively balancing efficiency and accuracy.

Technology Category

• Natural Language Processing: Question Answering
• Machine Learning: Large Multimodal Models (LMMs)
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Large language models for search

Abstract

Electronic health record (EHR) question answering is often handled by LLM-based pipelines that are costly to deploy and do not explicitly leverage the hierarchical structure of clinical data. Motivated by evidence that medical ontologies and patient trajectories exhibit hyperbolic geometry, we propose HypEHR, a compact Lorentzian model that embeds codes, visits, and questions in hyperbolic space and answers queries via geometry-consistent cross-attention with type-specific pointer heads. HypEHR is pretrained with next-visit diagnosis prediction and hierarchy-aware regularization to align representations with the ICD ontology. On two MIMIC-IV-based EHR-QA benchmarks, HypEHR approaches LLM-based methods while using far fewer parameters. Our code is publicly available at https://github.com/yuyuliu11037/HypEHR.

Problem

• Electronic Health Records
• Question Answering
• Hierarchical Structure
• Clinical Data
• LLM-based Pipelines

Innovation

• hyperbolic embedding
• electronic health records
• hierarchical modeling
• geometry-consistent attention
• ICD ontology alignment