Search papers, labs, and topics across Lattice.
This paper introduces LEED (Local Embedding Evolution Distance), a novel metric that quantifies over-smoothing in Graph Neural Networks (GNNs) by analyzing the evolution of individual node embeddings across layers. By providing a fine-grained, node-level analysis, LEED reveals heterogeneous over-smoothing patterns that traditional global metrics like Dirichlet energy miss, leading to more informative diagnostics and improved architectural decisions. The authors demonstrate that LEED not only enhances understanding of representation dynamics but also facilitates a more effective virtual node selection strategy, significantly boosting GNN performance across various datasets.
LEED uncovers hidden over-smoothing patterns in GNNs, enabling a new level of precision in node representation analysis and virtual node selection.
Graph Neural Networks (GNNs) suffer from two fundamental limitations: over-smoothing, where node representations become indistinguishable with depth, and over-squashing, where long-range information is compressed through limited message-passing channels. Existing metrics such as Dirichlet energy provide global characterizations of over-smoothing but lack the resolution to analyze node-level behavior and guide architectural improvements. In this paper, we propose LEED (Local Embedding Evolution Distance), a novel local metric that quantifies over-smoothing by tracking the evolution of individual node embeddings across layers. By operating at the node level, LEED enables fine-grained analysis of representation dynamics during training, revealing heterogeneous over-smoothing patterns that are invisible to global energy-based measures. This locality induces informative node importance scores, interpreted as embedding-driven centrality measures. We leverage LEED to design a more efficient strategy for virtual node selection. Unlike existing approaches that depend on multiple heuristic centrality measures, our method uses LEED as a unique criterion to guide the construction of Local Virtual Nodes to mitigate over-squashing. Experiments show that LEED provides more informative diagnostics than Dirichlet energy while preserving global evaluation, and enables more effective virtual node integration, improving GNN performance across datasets.
