Search papers, labs, and topics across Lattice.
This paper introduces Constrained Entity Selection under Partial Knowledge (CES-PK), a novel approach for knowledge graph question answering (KGQA) that leverages large language models (LLMs) to generate candidate answers, which are then verified using symbolic constraints derived from the question. By employing a three-valued constraint semantics to handle incomplete knowledge graphs, the method effectively filters out invalid answers while maintaining recall of potentially valid candidates. Experimental results on the Hetionet biomedical knowledge graph demonstrate a significant increase in precision without sacrificing recall, highlighting the effectiveness of the proposed framework in improving KGQA performance.
Filtering invalid answers with symbolic constraints can boost precision in knowledge graph QA while retaining valuable candidates.
Large language models are increasingly used for knowledge graph question answering (KGQA), but can fail to correctly ground answers in the underlying graph. Current approaches to LLM-based KGQA either rely on full semantic parsing into executable queries such as SPARQL, which is brittle in practice due to complex schemas or incompleteness of real-world KGs, or on LLM-reasoning and answer generation over KGs, which can be more robust but lacks formal guarantees. In this work, we study a complementary setting in which \emph{candidate} answers are generated by an LLM-based system and subsequently verified using lightweight symbolic constraints derived from the question. We introduce \emph{Constrained Entity Selection under Partial Knowledge (CES-PK)}, a problem formulation that focuses on eliminating invalid answers and providing symbolic support for valid ones without requiring construction of executable logical forms. To account for incomplete KGs, we employ a three-valued constraint semantics (\emph{satisfied, violated, unknown}) that avoids incorrect rejections under open-world assumptions. To demonstrate the effects of our method, we instantiate this framework over the Hetionet biomedical knowledge graph and evaluate the impact of type, relation, and exclusion constraints. Experiments show that precision improves by filtering invalid candidates, while recall is preserved due to retaining candidates whose constraints are not explicitly violated. Satisfied constraints provide additional positive symbolic evidence to rank remaining candidates.
