Learning from Disagreement: Clinician Overrides as Implicit Preference Signals for Clinical AI in Value-Based Care

2026-04-30

AI Summary

This work addresses the misalignment between clinical AI systems and patients’ long-term health goals, which often arises from overlooking physicians’ overrides of AI recommendations. Treating such override behaviors as implicit preference signals, the authors propose a conditional preference learning framework that jointly models patient state, organizational context, and physician capabilities, with a novel decomposition of execution and alignment capacities. By categorizing overrides into five types, employing a dual-model architecture—comprising a reward model and a capability model—and applying alternating optimization during training, the approach effectively mitigates suppression bias and prevents the systematic neglect of difficult yet correct recommendations. Built upon an extended RLHF paradigm augmented with longitudinal outcome supervision, the method demonstrates significant improvement in aligning AI recommendations with patients’ long-term health objectives in real-world value-based care settings.

Abstract

We reframe clinician overrides of clinical AI recommendations as implicit preference data - the same signal structure exploited by reinforcement learning from human feedback (RLHF), but richer: the annotator is a domain expert, the alternatives carry real consequences, and downstream outcomes are observable. We present a formal framework extending standard preference learning with three contributions: a five-category override taxonomy mapping override types to distinct model update targets; a preference formulation conditioned on patient state s, organizational context c, and clinician capability kappa, where kappa decomposes into execution capability kappa-exec and alignment capability kappa-align; and a dual learning architecture that jointly trains a reward model and a capability model via alternating optimization, preventing a failure mode we term suppression bias-the systematic suppression of correct-but-difficult recommendations when clinician capability falls below the execution threshold. We argue that chronic disease management under outcome-based payment contracts produces override data with uniquely favorable properties-longitudinal density, concentrated decision space, outcome labels, and natural capability variation-and that training environments combining longitudinal outcome measurement with aligned financial incentives are a necessary condition for learning a reward model aligned with patient trajectory rather than with encounter economics. This framework emerged from operational work to improve clinician capability in a live value-based care deployment.

Problem

• clinician overrides
• preference learning
• value-based care
• clinical AI
• reward modeling

Innovation

• preference learning
• clinician overrides
• capability modeling
• suppression bias
• value-based care