sebis at ArchEHR-QA 2026: How Much Can You Do Locally? Evaluating Grounded EHR QA on a Single Notebook

2026-03-14

AI Summary

This work addresses the challenges of privacy and computational constraints in deploying electronic health record (EHR) question-answering systems by participating in all four subtasks of ArchEHR-QA 2026 under a fully local setting—using only a single laptop without external APIs or cloud services. Leveraging lightweight language models and efficient local inference techniques, we developed an end-to-end EHR QA system that outperformed the average performance in two subtasks. Our results demonstrate that appropriately configured small-scale models can achieve performance comparable to larger systems on commodity hardware, effectively balancing privacy preservation with practical utility. This study validates the feasibility and competitiveness of localized EHR question-answering systems in resource-constrained clinical environments.

Technology Category

• Natural Language Processing: Question Answering
• Machine Learning: Hardware-aware ML
• Search and Optimization: Local Search

Application Category

• Search and Retrieval-Augmented AI: Web question answering
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• User Modeling, Personalization and Recommendation: User privacy protection in personalized systems

Abstract

Clinical question answering over electronic health records (EHRs) can help clinicians and patients access relevant medical information more efficiently. However, many recent approaches rely on large cloud-based models, which are difficult to deploy in clinical environments due to privacy constraints and computational requirements. In this work, we investigate how far grounded EHR question answering can be pushed when restricted to a single notebook. We participate in all four subtasks of the ArchEHR-QA 2026 shared task and evaluate several approaches designed to run on commodity hardware. All experiments are conducted locally without external APIs or cloud infrastructure. Our results show that such systems can achieve competitive performance on the shared task leaderboards. In particular, our submissions perform above average in two subtasks, and we observe that smaller models can approach the performance of much larger systems when properly configured. These findings suggest that privacy-preserving EHR QA systems running fully locally are feasible with current models and commodity hardware. The source code is available at https://github.com/ibrahimey/ArchEHR-QA-2026.

Problem

• EHR QA
• privacy-preserving
• local deployment
• clinical question answering
• commodity hardware

Innovation

• local EHR QA
• privacy-preserving AI
• commodity hardware
• grounded question answering
• small language models