Search papers, labs, and topics across Lattice.
This paper introduces the Structure-Internalized Rule Language Model (SIRLM), which addresses the challenge of reasoning evidence perception drift in Large Language Models (LLMs) when applied to Knowledge Graph Reasoning (KGR). By integrating a Structure-Internalized Rule Generator (SIRG) that combines structural rule generation with parametric learning and faithfulness evaluation, SIRLM enables LLMs to better align with the structural context of KGs. Experimental results show that SIRLM outperforms 17 state-of-the-art KGR methods across 36 datasets, highlighting its effectiveness in enhancing reasoning capabilities in KGR tasks.
SIRLM significantly boosts the reasoning fidelity of LLMs in Knowledge Graph tasks, outperforming existing methods by tightly coupling structural knowledge with parametric learning.
Knowledge Graph Reasoning (KGR) aims to discover latent facts by leveraging the structural evidence available in KGs, posing a challenge to the structural semantic understanding capability of KGR models. Recent studies have demonstrated that Large Language Models (LLMs) can achieve remarkable progress on KGR tasks via flexible in-context learning. However, the inherent representation inconsistency between KG structural context and LLM parametric knowledge remains inadequately addressed. This limitation prevents LLMs from effectively perceiving reasoning evidence that aligns with KG constraints, which undermines both the effectiveness and faithfulness of reasoning. We refer to this problem as reasoning evidence perception drift of LLMs over KGs. To address this problem, we propose a Structure-Internalized Rule Language Model (SIRLM), which centers on structural rule generation to couple the parametric learning of structural knowledge with the faithfulness evaluation of reasoning logic, enabling LLMs to anchor tightly to KG-grounded evidence. Specifically, we first design a Structure-Internalized Rule Generator (SIRG), which incorporates an in-context learning block augmented with a structural relation memory to coordinate structural and parametric knowledge. Furthermore, we equip SIRG with a KG tokenizer based on structural invariance learning and a neuro-symbolic reasoner based on rule-constrained message propagation. These components provide SIRG with learnable structural representations and faithful rule-execution feedback, respectively. Our SIRLM can be seamlessly integrated into standard LLM training paradigms, such as SFT and GRPO. Extensive experiments against 17 state-of-the-art KGR methods on 36 datasets demonstrate the significant superiority of SIRLM.
