Your Discrete Graph Diffusion Model is Secretly Memorizing

Key points

• In the unconditional setting without node features, an equivariant denoiser trained on a single graph is able to generate novel graphs and sample-level metrics such as edge overlap and WL kernel similarity find no evidence of memorization.1
• In this work, we challenge this view and show for the first time that graph diffusion models indeed exhibit memorization.1
• Analyzing the reverse diffusion trajectories of GDMs trained on two-block stochastic block models (SBMs), we find a strong train-data bias: the model recovers a corrupted training graph almost perfectly compared to a held-out test graph.1
• We term this conditional memorization, as it surfaces only when the reverse trajectory is conditioned on a corrupted training graph and is not apparent during inference.1

Sources (1)

• [1]Your Discrete Graph Diffusion Model is Secretly Memorizing
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 9, 04:04 PM

  Discrete graph diffusion models (GDMs) corrupt the graph structure by adding noise independently to nodes and edges under a Markovian noise model.

  In the unconditional setting without node features, an equivariant denoiser trained on a single graph is able to generate novel graphs and sample-level metrics such as edge overlap and WL kernel similarity find no evidence of memorization.

Extractive summary: sentences quoted from the sources.