Search papers, labs, and topics across Lattice.
This study investigates the integration of biomedical knowledge graphs (KGs) into large language models (LLMs) for clinical diagnosis, employing five KG task formulations across various training paradigms. The authors introduce two novel metrics, Gradient Intervention Density (GID) and Gradient Distortion (GD), to analyze the optimization behavior of LLMs during training, revealing that surgical alignment through KL regularization leads to sparse updates that enhance reasoning quality. Notably, even when in-domain accuracy is lower, paradigms that yield sparse updates outperform others in reasoning tasks, highlighting the importance of optimization geometry in assessing KG-LLM integration.
Sparse updates from surgical alignment can boost reasoning quality in LLMs, even when accuracy takes a hit.
Biomedical knowledge graphs (KGs) offer structured medical knowledge that can ground large language model (LLM) reasoning in clinical diagnosis application, yet how KG signal should be integrated into LLMs remains an open question. We present a systematic study spanning five KG task formulations, three training paradigms, two KGs, and three base LLMs. At the task level, all paradigms improve over the non-finetuned baseline, but methods with comparable in-domain accuracy show substantially different knowledge transfer behavior. We introduce Gradient Intervention Density (GID) and Gradient Distortion (GD) to measure how broadly an optimizer modifies the pretrained model. GID and GD together reveal a clear divide: KG-judgment training under KL regularization produces sparse, localized updates (a regime we term as surgical alignment), while task-specific SFT produces dense ones. A controlled ablation shows that the objective and KL contribute to sparsity independently, and the paradigms that produce sparse updates also improve reasoning quality, even when their in-domain accuracy is lower than task-specific SFT. Assessing KG-LLM integration thus requires complementing accuracy with optimization-geometry diagnostics. Our implementation can be found at https://github.com/LARK-NLP-Lab/Surgical-Alignment.
