Search papers, labs, and topics across Lattice.
This paper introduces an LLM-guided knowledge distillation framework to compress temporal knowledge graph (TKG) reasoning models. The framework uses a high-capacity temporal teacher model and an LLM to provide enriched supervision to a lightweight student model, capturing time-dependent interactions. Experiments on TKG benchmarks demonstrate that the approach improves link prediction performance over strong distillation baselines while maintaining a compact student model.
LLMs can serve as effective auxiliary teachers in knowledge distillation, significantly boosting the performance of lightweight temporal knowledge graph reasoning models.
Temporal knowledge graphs (TKGs) support reasoning over time-evolving facts, yet state-of-the-art models are often computationally heavy and costly to deploy. Existing compression and distillation techniques are largely designed for static graphs; directly applying them to temporal settings may overlook time-dependent interactions and lead to performance degradation. We propose an LLM-assisted distillation framework specifically designed for temporal knowledge graph reasoning. Beyond a conventional high-capacity temporal teacher, we incorporate a large language model as an auxiliary instructor to provide enriched supervision. The LLM supplies broad background knowledge and temporally informed signals, enabling a lightweight student to better model event dynamics without increasing inference-time complexity. Training is conducted by jointly optimizing supervised and distillation objectives, using a staged alignment strategy to progressively integrate guidance from both teachers. Extensive experiments on multiple public TKG benchmarks with diverse backbone architectures demonstrate that the proposed approach consistently improves link prediction performance over strong distillation baselines, while maintaining a compact and efficient student model. The results highlight the potential of large language models as effective teachers for transferring temporal reasoning capability to resource-efficient TKG systems.
