Search papers, labs, and topics across Lattice.
This paper investigates the zero-shot size-transfer capability of Graph Neural Differential Equations (GNDEs) when applied to sparse random graphs, establishing a quantitative theory for their deployment from small to larger graphs without retraining. The authors prove that GNDE solutions converge to Graphon-NDE solutions at a rate of \(O((\alpha_n n)^{-1/2})\) with high probability, and provide uniform convergence bounds for adjoint systems that govern hidden-state and parameter gradients. Experimental results validate the theoretical findings, showcasing effective zero-shot transfer across multiple graphon classes, indicating the robustness of the proposed methods in practical applications.
Zero-shot size transfer allows GNDEs trained on small graphs to accurately model larger graphs, revolutionizing scalability in graph-based learning.
Graph Neural Differential Equations (GNDEs) model continuous-time graph dynamics by parameterizing Neural ODE velocity fields with Graph Neural Networks. Their local, size-independent filters suggest a zero-shot size-transfer principle: train on a small graph and deploy on larger, similar graphs without retraining. We develop a quantitative theory for this principle on sparse random graphs sampled from graphons. We consider Graphon Neural Differential Equations (Graphon-NDEs) and adjoint Graphon-NDEs as the infinite-node limits of the forward and adjoint GNDE systems, and establish well-posedness. For an $n$-node random graph with sparsity parameter $伪_n$, we prove trajectory-wise convergence of GNDE solutions to Graphon-NDE solutions at rate $O((伪_n n)^{-1/2})$, up to logarithmic factors, with high probability. We also establish uniform-in-time convergence bounds for adjoint systems governing hidden-state and parameter gradients. We further study discretize-then-optimize (DTO) and optimize-then-discretize (OTD) training. Under explicit Euler discretization with $M$ steps, we show that DTO and OTD are asymptotically consistent, with hidden-state and local parameter-gradient discrepancies of orders $O(1/M)$ and $O(1/M^2)$, respectively, up to sparsity and logarithmic factors. Experiments on HSBM and tent graphons support the theoretical rates, while zero-shot transfer experiments across four graphon classes demonstrate accurate deployment of learned GNDEs on larger independently sampled graphs.
