Search papers, labs, and topics across Lattice.
This paper investigates the relationship between graph topology, specifically curvature, and activation patterns in Graph Neural Networks (GNNs) by analyzing Massive Activations (MAs) in Graph Transformers. The study reveals that MAs do not preferentially concentrate on curvature extremes in synthetic and molecular graphs, challenging the direct link between information flow and topological bottlenecks. Furthermore, the authors identify a "curvature shift" on the Long Range Graph Benchmark, where global attention mechanisms exacerbate topological bottlenecks, leading to increased negative curvature.
GNNs don't "see" graph curvature the way theory predicts, and global attention can actually *worsen* the problem by amplifying topological bottlenecks.
Curvature notions on graphs provide a theoretical description of graph topology, highlighting bottlenecks and denser connected regions. Artifacts of the message passing paradigm in Graph Neural Networks, such as oversmoothing and oversquashing, have been attributed to these regions. However, it remains unclear how the topology of a graph interacts with the learned preferences of GNNs. Through Massive Activations, which correspond to extreme edge activation values in Graph Transformers, we probe this correspondence. Our findings on synthetic graphs and molecular benchmarks reveal that MAs do not preferentially concentrate on curvature extremes, despite their theoretical link to information flow. On the Long Range Graph Benchmark, we identify a systemic \textit{curvature shift}: global attention mechanisms exacerbate topological bottlenecks, drastically increasing the prevalence of negative curvature. Our work reframes curvature as a diagnostic probe for understanding when and why graph learning fails.
