Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper reveals that discrete graph diffusion models (GDMs), despite appearing to generate novel graphs and showing no evidence of memorization via standard metrics, exhibit significant conditional memorization. By analyzing reverse diffusion trajectories conditioned on corrupted training graphs, the authors demonstrate a strong bias towards recovering training data over generating novel graphs, a phenomenon that worsens generalization and precedes the emergence of meaningful graph structure. This memorization is robust across tested node sizes and cannot be mitigated by early stopping.
Discrete graph diffusion models, believed to generate novel graphs, are actually memorizing training data, with this bias appearing only when conditioned on corrupted training examples.
Discrete graph diffusion models (GDMs) corrupt the graph structure by adding noise independently to nodes and edges under a Markovian noise model. In the unconditional setting without node features, an equivariant denoiser trained on a single graph is able to generate novel graphs and sample-level metrics such as edge overlap and WL kernel similarity find no evidence of memorization. In this work, we challenge this view and show for the first time that graph diffusion models indeed exhibit memorization. Analyzing the reverse diffusion trajectories of GDMs trained on two-block stochastic block models (SBMs), we find a strong train-data bias: the model recovers a corrupted training graph almost perfectly compared to a held-out test graph. We term this conditional memorization, as it surfaces only when the reverse trajectory is conditioned on a corrupted training graph and is not apparent during inference. It coincides with a growing generalization gap, where the training loss improves as the loss on held-out graphs degrades, and it cannot be mitigated by early stopping. The onset of memorization always precedes the emergence of community structure in the generated graphs, thus every checkpoint earlier produces structureless graphs. This behavior holds across all node sizes we test.