Search papers, labs, and topics across Lattice.
The paper introduces MINAR, a toolbox that adapts attribution patching from mechanistic interpretability to graph neural networks (GNNs) to discover neuron-level circuits in neural algorithmic reasoning (NAR) models. Using MINAR, the authors identify circuits in GNNs trained on algorithmic tasks, specifically focusing on how these circuits form and are pruned during training. The study also reveals how GNNs reuse circuit components when trained to perform multiple related tasks in parallel.
Uncovers how GNNs internally implement and reuse algorithmic steps, paving the way for understanding and controlling neural algorithmic reasoning.
The recent field of neural algorithmic reasoning (NAR) studies the ability of graph neural networks (GNNs) to emulate classical algorithms like Bellman-Ford, a phenomenon known as algorithmic alignment. At the same time, recent advances in large language models (LLMs) have spawned the study of mechanistic interpretability, which aims to identify granular model components like circuits that perform specific computations. In this work, we introduce Mechanistic Interpretability for Neural Algorithmic Reasoning (MINAR), an efficient circuit discovery toolbox that adapts attribution patching methods from mechanistic interpretability to the GNN setting. We show through two case studies that MINAR recovers faithful neuron-level circuits from GNNs trained on algorithmic tasks. Our study sheds new light on the process of circuit formation and pruning during training, as well as giving new insight into how GNNs trained to perform multiple tasks in parallel reuse circuit components for related tasks. Our code is available at https://github.com/pnnl/MINAR.
Google Research