Surgical Alignment in Knowledge Graph Training for Clinical Diagnosis with Large Language Models

2026-08-26

AI Summary

研究通过五种任务形式、三种训练范式等方法，解决如何将生物医学知识图谱信号有效整合到大型语言模型中的问题，并提出使用梯度干预密度和梯度失真作为评估指标。

Abstract

Biomedical knowledge graphs (KGs) offer structured medical knowledge that can ground large language model (LLM) reasoning in clinical diagnosis application, yet how KG signal should be integrated into LLMs remains an open question. We present a systematic study spanning five KG task formulations, three training paradigms, two KGs, and three base LLMs. At the task level, all paradigms improve over the non-finetuned baseline, but methods with comparable in-domain accuracy show substantially different knowledge transfer behavior. We introduce Gradient Intervention Density (GID) and Gradient Distortion (GD) to measure how broadly an optimizer modifies the pretrained model. GID and GD together reveal a clear divide: KG-judgment training under KL regularization produces sparse, localized updates (a regime we term as surgical alignment), while task-specific SFT produces dense ones. A controlled ablation shows that the objective and KL contribute to sparsity independently, and the paradigms that produce sparse updates also improve reasoning quality, even when their in-domain accuracy is lower than task-specific SFT. Assessing KG-LLM integration thus requires complementing accuracy with optimization-geometry diagnostics. Our implementation can be found at https://github.com/LARK-NLP-Lab/Surgical-Alignment.

Problem

• Knowledge Graphs
• Large Language Models
• Clinical Diagnosis
• Integration
• Training Paradigms

Innovation

• Surgical Alignment
• Gradient Intervention Density (GID)
• Gradient Distortion (GD)
• Knowledge Graph (KG)
• Large Language Models (LLMs)