Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper identifies spectral underfitting and overfitting as key failure modes in GNN-to-MLP distillation, stemming from the omission of graph geometry transfer. To address this, they propose Graph Geometry-aware MLP (G^2MLP), a framework that leverages Ollivier-Ricci curvature to guide representation-level alignment alongside prediction-level distillation. G^2MLP demonstrably improves performance across node-classification and link-prediction tasks by effectively mitigating these spectral errors without requiring graph access at inference.
Ollivier-Ricci curvature reveals and rectifies spectral representation errors in GNN-to-MLP distillation, unlocking significant accuracy gains for graph-free student models.
GNN-to-MLP distillation aims to retain the predictive accuracy of a message-passing teacher while deploying a graph-free MLP at inference. Existing methods mainly transfer node-wise predictions or use confidence-based reweighting, but they do not specify where the student should preserve the teacher's graph-induced geometry. We show that this omission leads to two spectral failure modes in the student's representation space. On sparse graphs, the student suffers from spectral underfit, missing high-energy teacher directions concentrated near boundary regions. On dense graphs, it suffers from spectral overfit, retaining spurious directions that the teacher has collapsed through aggregation. Motivated by an energy-weighted teacher-student alignment objective, we propose Graph Geometry-aware MLP (G^2MLP), a training-time distillation framework guided by Ollivier-Ricci curvature. Curvature identifies where the two spectral errors concentrate and is used to allocate supervision between prediction-level and representation-level alignment. The deployed model remains a standard MLP and requires no graph access at inference. Across node-classification benchmarks, G^2MLP consistently improves over graph-free distillation baselines, reduces the teacher-student rank gap in both regimes, and transfers without architectural changes to Graph Transformer teachers and link prediction.