Training-Free Adaptation of New-Generation LLMs using Legacy Clinical Models

2026-01-06

arXiv.org

AI Summary

This work addresses the challenge of deploying next-generation general-purpose large language models in clinical settings, where prohibitive retraining costs often hinder effective integration with existing specialized models. To overcome this limitation, the authors propose Cross-architecture Proxy Tuning (CAPT), a novel method that, for the first time, enables seamless fusion of general and clinical language models across different generations and vocabularies without any training. CAPT leverages contrastive decoding to align vocabulary-agnostic signals, thereby injecting clinical knowledge while preserving the general model’s reasoning capabilities and linguistic fluency. Evaluated on six clinical classification and generation tasks, CAPT substantially outperforms UniTE by 17.6% and proxy tuning by 41.4% on average, while also enhancing the clinical actionability and terminological specificity of generated outputs.

Technology Category

• Natural Language Processing: (Large) Language Models
• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Large Vision Models

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Adapting language models to the clinical domain through continued pretraining and fine-tuning requires costly retraining for each new model generation. We propose Cross-Architecture Proxy Tuning (CAPT), a model-ensembling approach that enables training-free adaptation of state-of-the-art general-domain models using existing clinical models. CAPT supports models with disjoint vocabularies, leveraging contrastive decoding to selectively inject clinically relevant signals while preserving the general-domain model's reasoning and fluency. On six clinical classification and text-generation tasks, CAPT with a new-generation general-domain model and an older-generation clinical model consistently outperforms both models individually and state-of-the-art ensembling approaches (average +17.6% over UniTE, +41.4% over proxy tuning across tasks). Through token-level analysis and physician case studies, we demonstrate that CAPT amplifies clinically actionable language, reduces context errors, and increases clinical specificity.

Problem

• clinical adaptation
• large language models
• training-free
• model ensembling
• domain adaptation

Innovation

• Cross-Architecture Proxy Tuning
• training-free adaptation
• clinical language models
• contrastive decoding
• model ensembling