Search papers, labs, and topics across Lattice.
The paper introduces Explore-on-Graph (EoG), a reinforcement learning framework that trains LLMs to autonomously explore knowledge graphs for question answering, addressing limitations of existing methods that constrain reasoning to prior knowledge. EoG uses a reward function based on the correctness of reasoning paths' final answers, refined with path information to guide exploration. Experiments on five KGQA datasets demonstrate that EoG achieves state-of-the-art performance, surpassing both open-source and closed-source LLMs.
LLMs can now explore knowledge graphs on their own, discovering better reasoning paths and outperforming even closed-source models on question answering.
The reasoning process of Large Language Models (LLMs) is often plagued by hallucinations and missing facts in question-answering tasks. A promising solution is to ground LLMs' answers in verifiable knowledge sources, such as Knowledge Graphs (KGs). Prevailing KG-enhanced methods typically constrained LLM reasoning either by enforcing rules during generation or by imitating paths from a fixed set of demonstrations. However, they naturally confined the reasoning patterns of LLMs within the scope of prior experience or fine-tuning data, limiting their generalizability to out-of-distribution graph reasoning problems. To tackle this problem, in this paper, we propose Explore-on-Graph (EoG), a novel framework that encourages LLMs to autonomously explore a more diverse reasoning space on KGs. To incentivize exploration and discovery of novel reasoning paths, we propose to introduce reinforcement learning during training, whose reward is the correctness of the reasoning paths' final answers. To enhance the efficiency and meaningfulness of the exploration, we propose to incorporate path information as additional reward signals to refine the exploration process and reduce futile efforts. Extensive experiments on five KGQA benchmark datasets demonstrate that, to the best of our knowledge, our method achieves state-of-the-art performance, outperforming not only open-source but also even closed-source LLMs.
