Search papers, labs, and topics across Lattice.
This paper investigates the vulnerabilities of GNN-based Knowledge Graph Question Answering (KGQA) systems by isolating failures across four processing stages: entity linking, subgraph retrieval, GNN reasoning, and answer generation. Through a novel stage-isolation protocol employing adversarial perturbations, the authors reveal that over 99% of performance degradation stems from subgraph construction, even when the correct answer is present in the retrieved subgraphs. This finding challenges existing assumptions about the robustness of GNN reasoning and highlights the need for targeted improvements in subgraph retrieval processes.
Subgraph construction, not GNN reasoning, is the critical failure point in KGQA systems, accounting for over 99% of performance collapse under adversarial conditions.
GNN-based Knowledge Graph Question Answering (KGQA) pipelines process queries through four discrete stages: entity linking, subgraph retrieval, GNN reasoning, and answer generation. Standard robustness evaluations conflate stage-level failures into a single end-to-end metric, obscuring both the source of brittleness and the appropriate mitigation target. We ask which stage fails, and why, when the pipeline is subjected to adversarial perturbations on the input question. We introduce a stage-isolation protocol with two answer-preserving adversarial perturbations verified against the knowledge graph: Compositional Restructuring (CR) and Relation Synonym Swap (RS) target distinct stages while leaving entity seeds intact. Evaluated across ComplexWebQuestions and WebQSP, the results run counter to prevailing assumptions: the GNN reasoning stage retains near-baseline accuracy when the subgraph is intact, while subgraph construction accounts for over 99\% of the end-to-end collapse under CR, occurring even when the gold answer is present in 74\% of retrieved subgraphs. This exposes a fundamental distinction between answer presence and answer reachability that end-to-end metrics cannot detect, and places the mitigation target firmly at the subgraph construction stage rather than the reasoning model. Perturbed datasets and evaluation infrastructure are released at https://anonymous.4open.science/r/atkgrag-E85C .
