Teaching LLMs to Recommend and Defer in Underrepresented Epilepsy Care

2026-06-29

AI Summary

This study addresses the critical gap in epilepsy care at the primary level in resource-constrained Uganda, where clinical decision support tools are lacking that align with local prescribing practices and reliably identify when to refer patients. The authors propose MANANA, a non-parametric prompt-learning framework grounded in large language models, which employs a multi-agent prompting architecture to learn local prescription patterns and a Bayesian prompt averaging mechanism to generate uncertainty-aware referral signals. Requiring only limited patient-level data for localization, MANANA improves top-3 prescription accuracy by 4–8 percentage points on held-out test cases. The system achieves 95% precision on 50% of cases (or 99% on 25%), automatically referring low-confidence cases to specialists, thereby enabling high-precision selective prediction with a safety net for uncertain diagnoses.

Abstract

Specialist epilepsy expertise is scarce in resource-constrained settings, making LLM-based decision support attractive for frontline clinicians managing longitudinal treatment. Such systems must adapt to local prescribing practice and know when to defer. We study this problem in Ugandan pediatric epilepsy care, predicting anti-seizure medication regimens from longitudinal unstructured clinic notes. Standard prompting achieves non-trivial agreement with physician prescriptions, but neurologist review shows that many errors reflect distribution-miscalibrated prescribing defaults rather than failures to parse the local record. We introduce MANANA, a non-parametric prompt-learning framework that learns local prescribing guidance from a small patient-level training set. MANANA converts observed prescription errors into auditable prompt memories, instantiated in single-agent and multi-agent variants, and improves over classical ML models, direct LLM prompting, and prompt-optimization baselines across two independently collected Ugandan cohorts. We further propose Bayesian prompt averaging, which converts the learned prompt trajectory into prescription likelihoods and an uncertainty-based deferral signal. On the independently collected held-out cohort, this improves visit-level top-3 prescription accuracy by 4-8 percentage points over prompt-optimization baselines and enables selective prediction: the system can auto-handle the most confident half of cases at 95% precision, or the most confident quarter at 99% precision, while deferring lower-confidence cases for specialist review.

Problem

• epilepsy care
• LLM recommendation
• clinical decision support
• treatment deferral
• resource-constrained settings

Innovation

• prompt learning
• selective prediction
• Bayesian prompt averaging
• LLM deferral
• epilepsy care