From Fuzzy to Formal: Scaling Hospital Quality Improvement with AI

2026-04-21

AI Summary

Hospital quality improvement traditionally relies on expert-driven qualitative analysis, which is time-consuming, inefficient, and difficult to reproduce. This work proposes a “human–AI norm–solution co-optimization” framework that formalizes ambiguous expert judgments into tunable natural language norm hyperparameters. By integrating large language model prompt learning with classical AI/ML pipelines—including problem formalization, model learning, and validation—the approach enables systematic and automated exploratory analysis of healthcare quality. Evaluated in an urban safety-net hospital setting, the method achieves over 70% agreement with expert annotations and substantially outperforms conventional Lean analysis. It not only replicates known findings but also uncovers novel modifiable factors and produces auditable reasoning traces, thereby enhancing both scalability and transparency in quality improvement initiatives.

Technology Category

• Search and Optimization: Learning to Search
• Humans and AI: Learning Human Values and Preferences
• Natural Language Processing: Safety and Robustness

Application Category

• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Hospital Quality Improvement (QI) plays a critical role in optimizing healthcare delivery by translating high-level hospital goals into actionable solutions. A critical step of QI is to identify the key modifiable contributing factors, a process we call QI factor discovery, typically through expert-driven semi-structured qualitative tools like fishbone diagrams, chart reviews, and Lean Healthcare methods. AI has the potential to transform and accelerate QI factor discovery, which is traditionally time- and resource-intensive and limited in reproducibility and auditability. Nevertheless, current AI alignment methods assume the task is well-defined, whereas QI factor discovery is an exploratory, fuzzy, and iterative sense-making process that relies on complex implicit expert judgments. To design an AI pipeline that formalizes the QI process while preserving its exploratory components, we propose viewing the task as learning not only LLM prompts but also the overarching natural-language specifications. In particular, we map QI factor discovery to steps of the classical AI/ML development process (problem formalization, model learning, and model validation) where the specifications are tunable hyperparameters. Domain experts and AI agents iteratively refine both the overarching specifications and AI pipeline until AI extractions are concordant with expert annotations and aligned with clinical objectives. We applied this "Human-AI Spec-Solution Co-optimization" framework at an urban safety-net hospital to identify factors driving prolonged length of stay and unplanned 30-day readmissions. The resulting AI-for-QI pipelines achieved $\ge 70\%$ concordance with expert annotations. Compared to prior manual Lean analyses, the AI pipeline was substantially more efficient, recovered previous findings, surfaced new modifiable factors, and produced auditable reasoning traces.

Problem

• Hospital Quality Improvement
• QI factor discovery
• AI alignment
• exploratory process
• expert judgment

Innovation

• Human-AI co-optimization
• specification learning
• quality improvement
• LLM prompting
• interpretable AI