ClinAlign: Scaling Healthcare Alignment from Clinician Preference

2026-02-10

AI Summary

Although large language models possess expert-level medical knowledge, their open-ended outputs struggle to align precisely with clinicians’ fine-grained preferences, and existing alignment methods lack reliable evaluation mechanisms grounded in professional clinical guidelines. To address this, this work proposes a two-stage framework: first, constructing HealthRubrics—a dataset of 7,034 clinician-validated preference pairs—and then distilling it into 119 reusable HealthPrinciples that guide both offline alignment and on-the-fly model self-correction during inference. This approach establishes the first structured, scalable clinical principle system derived directly from real physician preferences. By integrating synthetic scoring rules with a sparsely activated large language model (30B parameters with only 3B active), the method achieves 33.4% performance on HealthBench-Hard, surpassing larger models such as Deepseek-R1 and o3, thereby setting a new efficient and generalizable baseline for medical alignment.

Technology Category

• Natural Language Processing: (Large) Language Models
• Machine Learning: Learning Preferences or Rankings
• Humans and AI: Learning Human Values and Preferences

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Although large language models (LLMs) demonstrate expert-level medical knowledge, aligning their open-ended outputs with fine-grained clinician preferences remains challenging. Existing methods often rely on coarse objectives or unreliable automated judges that are weakly grounded in professional guidelines. We propose a two-stage framework to address this gap. First, we introduce HealthRubrics, a dataset of 7,034 physician-verified preference examples in which clinicians refine LLM-drafted rubrics to meet rigorous medical standards. Second, we distill these rubrics into HealthPrinciples: 119 broadly reusable, clinically grounded principles organized by clinical dimensions, enabling scalable supervision beyond manual annotation. We use HealthPrinciples for (1) offline alignment by synthesizing rubrics for unlabeled queries and (2) an inference-time tool for guided self-revision. A 30B-A3B model trained with our framework achieves 33.4% on HealthBench-Hard, outperforming much larger models including Deepseek-R1 and o3, establishing a resource-efficient baseline for clinical alignment.

Problem

• clinical alignment
• clinician preference
• large language models
• medical guidelines
• healthcare AI

Innovation

• ClinAlign
• HealthRubrics
• HealthPrinciples
• clinical alignment
• preference distillation