TriageRA-CCF: Source-Side Clinical Confidence and Coverage Signals for Adaptive Rank Budgeting in Medical LLMs

2026-06-28

AI Summary

This work addresses the limitation of existing medical large language models that employ a fixed low-rank budget in question-answering tasks, preventing dynamic allocation of computational resources according to the clinical confidence, coverage, and difficulty of individual queries. To overcome this, the authors propose a source-driven adaptive rank budgeting method that, for the first time, integrates model confidence, clinical coverage, and counterfactual near-miss signals from source data to construct a self-supervised budget routing mechanism without requiring target-domain information. Built upon the LoRA architecture, the approach incorporates a straight-through router, entropy regularization, budget cost constraints, and a rank-balancing loss, trained exclusively on source-domain data. Evaluated on Qwen3-8B and Llama3.1-8B, it consistently outperforms LoRA, DoRA, and MoELoRA baselines, achieving average accuracy gains of 0.21 and 0.16 percentage points, respectively.

Abstract

Medical large language models are commonly adapted with a fixed low-rank budget, even though medical questions differ substantially in confidence, clinical coverage, and cross-domain difficulty. We study adaptive rank budgeting for parameter-efficient medical question answering: for each question, the adapter decides whether to activate a small, medium, or large subset of LoRA rank channels. The central challenge is that a naive adaptive budget router can collapse to unstable choices or spend capacity without improving shifted benchmarks. We propose TriageRA-CCF, a source-side teacher for adaptive rank-budgeted LoRA. It combines three signals computed only from source training data: base-model answer confidence, metadata-cell clinical coverage, and a counterfactual close-miss proxy. These signals supervise a straight-through budget router over active ranks {2,4,8}, together with budget-cost, entropy, and rank-balance regularization. Under a matched CMB-source training protocol, TriageRA-CCF achieves the best average accuracy among LoRA, DoRA, and MoELoRA baselines on both Qwen3-8B and Llama3.1-8B. The gains are modest and non-uniform across benchmarks: +0.21 average points over the strongest external baseline on Qwen3-8B and +0.16 on Llama3.1-8B. Component ablations show that confidence, coverage, and counterfactual signals all provide useful budget supervision, but their combination is not monotonically best on every backbone.

Problem

• adaptive rank budgeting
• medical LLMs
• LoRA
• clinical confidence
• parameter-efficient fine-tuning

Innovation

• adaptive rank budgeting
• LoRA
• clinical confidence
• counterfactual proxy
• parameter-efficient fine-tuning