Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
The paper identifies and addresses two key biases in graph diffusion models: reverse-starting bias, caused by a mismatch between the forward diffusion's maximum perturbation distribution and the reverse sampling's Gaussian starting point, and exposure bias. They mitigate reverse-starting bias using a novel Langevin sampling algorithm to align the reverse starting point with the forward perturbation distribution, and address exposure bias with a score correction mechanism based on a defined score difference. Experiments across various models, datasets, and tasks demonstrate state-of-the-art performance without requiring network modifications.
Graph diffusion models suffer from a reverse-starting bias that can be fixed with a simple Langevin sampling tweak, leading to SOTA results without any network changes.
Most existing graph diffusion models have significant bias problems. We observe that the forward diffusion's maximum perturbation distribution in most models deviates from the standard Gaussian distribution, while reverse sampling consistently starts from a standard Gaussian distribution, which results in a reverse-starting bias. Together with the inherent exposure bias of diffusion models, this results in degraded generation quality. This paper proposes a comprehensive approach to mitigate both biases. To mitigate reverse-starting bias, we employ a newly designed Langevin sampling algorithm to align with the forward maximum perturbation distribution, establishing a new reverse-starting point. To address the exposure bias, we introduce a score correction mechanism based on a newly defined score difference. Our approach, which requires no network modifications, is validated across multiple models, datasets, and tasks, achieving state-of-the-art results.Code is at https://github.com/kunzhan/spp