PVminerLLM2: Improving Structured Extraction of Patient Voice via Preference Optimization

2026-06-14

AI Summary

Patient-generated text contains rich experiential and sociocultural information, yet its unstructured nature and token-level labeling errors—such as rarity, fine-grained distinctions, and class imbalance—hinder its utility in patient-centered research. This work proposes a preference optimization framework tailored for structured information extraction, introducing a novel token-level gating stability term to prevent likelihood degradation, designing ambiguity-aware preference pairs to enhance recognition of low-discriminability labels, and integrating token importance weighting with inverse frequency reweighting to mitigate class imbalance. Experimental results demonstrate consistent improvements over strong baselines across multiple model scales, achieving gains of 4.43%, 3.50%, and 1.55% on Code, Sub-code, and Span metrics, respectively, substantially outperforming existing preference optimization approaches.

Abstract

Motivation: Patient-generated text contains critical information on patients' lived experiences, social context, and care engagement, but remains largely unstructured, limiting its use in patient-centered outcomes research. Prior work introduced the PV-Miner benchmark and PVMinerLLM models for structured extraction. However, supervised fine-tuning (SFT) alone struggles with rare, fine-grained, and unevenly distributed errors, particularly in token-critical structured outputs. Results: We present PVminerLLM2, an improved set of LLMs for structured patient voice extraction that applies preference optimization to address token-critical errors beyond the reach of supervised fine-tuning. Our method introduces (i) a preference objective with token-level gated stabilization term that prevents degradation of absolute token likelihood under preference optimization, and (ii) confusion-aware preference pair construction to better capture low-separation distinctions. We further incorporate token-importance weighting and inverse-frequency reweighing to address token imbalance and class skew. Across multiple model sizes, PVMinerLLM2 consistently outperforms strong baselines, achieving gains of up to 4.43% (Code), 3.50% (Sub-code), and 1.55% (Span), and outperforms baseline LLM trained with existing preference optimization methods. Availability and Implementation: The supplementary material, code, evaluation scripts, and trained models for PVminerLLM2 are publicly available at: https://github.com/Data-Mining-Lab-Yale/PVminerLLM2

Problem

• structured extraction
• patient voice
• token-critical errors
• preference optimization
• LLM

Innovation

• preference optimization
• token-level stabilization
• confusion-aware preference
• structured extraction
• patient voice