Search papers, labs, and topics across Lattice.
This paper introduces NEST-KGQA, a new knowledge graph question answering task and dataset focused on questions with negative constraints, an area often neglected in existing benchmarks. To address the challenges posed by negative and multiple constraints, they propose CUCKOO, a framework that generates Python-formatted logical forms (PyLF), performs schema-guided semantic matching, and applies self-directed refinement when initial logical form execution yields empty results. Experiments demonstrate that CUCKOO outperforms baselines on both conventional KGQA and the new NEST-KGQA benchmark in few-shot settings.
LLMs still struggle with questions containing negative constraints, but a new framework leveraging Python-formatted logical forms and self-directed refinement significantly improves performance on this neglected but common real-world KGQA task.
Large language models still struggle with faithfulness and hallucinations despite their remarkable reasoning abilities. In Knowledge Graph Question Answering (KGQA), semantic parsing-based approaches address the limitations by understanding constraints in a user's question and converting them into a logical form to execute on a knowledge graph. However, existing KGQA benchmarks and methods are biased toward positive and calculation constraints. Negative constraints are neglected, although they frequently appear in real-world questions. In this paper, we introduce a new task, NEgative-conSTrained (NEST) KGQA, where each question contains at least one negative constraint, and a corresponding dataset, NestKGQA. We also design PyLF, a Python-formatted logical form, since existing logical forms are hardly suitable to express negation clearly while maintaining readability. Furthermore, NEST questions naturally contain multiple constraints. To mitigate their semantic complexity, we present a novel framework named CUCKOO, specialized to multiple-constrained questions and ensuring semantic executability. CUCKOO first generates a constraint-aware logical form draft and performs schema-guided semantic matching. It then selectively applies self-directed refinement only when executing improper logical forms yields an empty result, reducing cost while improving robustness. Experimental results demonstrate that CUCKOO consistently outperforms baselines on both conventional KGQA and NEST-KGQA benchmarks under few-shot settings.
