Search papers, labs, and topics across Lattice.
The paper introduces interpolated FID (iFID), a modification of reconstruction FID (rFID) that strongly correlates with the generation FID (gFID) of latent diffusion models. iFID involves interpolating latent representations of data points with their nearest neighbors in latent space before decoding and computing FID. The authors demonstrate that iFID correlates with sample quality in the diffusion navigation phase, while rFID correlates with the refinement phase, and provide theoretical justification based on diffusion generalization and hallucination. Empirically, iFID achieves a Pearson linear and Spearman rank correlation of approximately 0.85 with diffusion gFID.
Interpolating latent representations before decoding yields a reconstruction FID (iFID) that finally aligns with the generation FID of latent diffusion models, achieving ~0.85 correlation where standard rFID fails.
It is well known that the reconstruction FID (rFID) of a VAE is poorly correlated with the generation FID (gFID) of a latent diffusion model. We propose interpolated FID (iFID), a simple variant of rFID that exhibits a strong correlation with gFID. Specifically, for each element in the dataset, we retrieve its nearest neighbor (NN) in the latent space and interpolate their latent representations. We then decode the interpolated latent and compute the FID between the decoded samples and the original dataset. Additionally, we refine the claim that rFID correlates poorly with gFID, by showing that rFID correlates with sample quality in the diffusion refinement phase, whereas iFID correlates with sample quality in the diffusion navigation phase. Furthermore, we provide an explanation for why iFID correlates well with gFID, and why reconstruction metrics are negatively correlated with gFID, by connecting to results in the diffusion generalization and hallucination. Empirically, iFID is the first metric to demonstrate a strong correlation with diffusion gFID, achieving Pearson linear and Spearman rank correlations approximately 0.85. The source code is provided in https://github.com/tongdaxu/Making-rFID-Predictive-of-Diffusion-gFID.
