Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This study critically examines the ability of large language models (LLMs) to understand graph isomorphism, a key concept in graph theory. Despite achieving near-perfect accuracy in detecting isomorphic graphs, LLMs falter when faced with graphs that have permuted node labels, revealing that their performance relies on pattern recognition rather than true structural reasoning. This finding underscores the misconception that high accuracy in graph reasoning tasks equates to a genuine understanding of graph topology, challenging the interpretation of LLM capabilities in this domain.
LLMs may excel at detecting isomorphism, but they completely miss the mark when node labels are permuted, exposing a critical flaw in their structural reasoning abilities.
Large language models (LLMs) have shown impressive performance on diverse reasoning tasks, yet their capacity for structural reasoning in graphs remains unclear. We investigate whether LLMs can genuinely understand graph isomorphism -a fundamental problem in graph theory. While LLMs achieve near-perfect accuracy on isomorphism detection, we show this performance is illusory. When identical graphs are presented with permuted node labels, LLMs fail to identify their isomorphism. This finding suggests that LLMs exploit patterns rather than reasoning about abstract graph structure. Since permutation invariance is a fundamental requirement for valid structural reasoning, these results indicate that success on graph reasoning benchmarks should not be interpreted as evidence of genuine topological understanding.