Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper investigates the vulnerabilities of GNN-based Knowledge Graph Question Answering (KGQA) systems by isolating failures across four processing stages: entity linking, subgraph retrieval, GNN reasoning, and answer generation. Through a novel stage-isolation protocol using two types of adversarial perturbations, the study reveals that the subgraph construction stage is responsible for over 99% of performance degradation, even when the correct answer is present in a majority of retrieved subgraphs. These findings challenge existing evaluation methods that conflate stage-level failures and highlight the need for targeted improvements in subgraph construction to enhance overall system robustness.
Subgraph construction, not GNN reasoning, is the Achilles' heel of KGQA systems, accounting for over 99% of performance collapse under adversarial conditions.
GNN-based Knowledge Graph Question Answering (KGQA) pipelines process queries through four discrete stages: entity linking, subgraph retrieval, GNN reasoning, and answer generation. Standard robustness evaluations conflate stage-level failures into a single end-to-end metric, obscuring both the source of brittleness and the appropriate mitigation target. We ask which stage fails, and why, when the pipeline is subjected to adversarial perturbations on the input question. We introduce a stage-isolation protocol with two answer-preserving adversarial perturbations verified against the knowledge graph: Compositional Restructuring (CR) and Relation Synonym Swap (RS) target distinct stages while leaving entity seeds intact. Evaluated across ComplexWebQuestions and WebQSP, the results run counter to prevailing assumptions: the GNN reasoning stage retains near-baseline accuracy when the subgraph is intact, while subgraph construction accounts for over 99\% of the end-to-end collapse under CR, occurring even when the gold answer is present in 74\% of retrieved subgraphs. This exposes a fundamental distinction between answer presence and answer reachability that end-to-end metrics cannot detect, and places the mitigation target firmly at the subgraph construction stage rather than the reasoning model. Perturbed datasets and evaluation infrastructure are released at https://anonymous.4open.science/r/atkgrag-E85C .