Making Reconstruction FID Predictive of Diffusion Generation FID

2026-03-05

AI Summary

This work addresses the poor correlation between reconstruction-based FID (rFID) and generation-based FID (gFID) in diffusion models, which limits the effectiveness of rFID for evaluating generative quality. To bridge this gap, the authors propose interpolated FID (iFID), which computes FID on samples decoded from interpolations between latent representations and their nearest neighbors. iFID achieves a strong correlation with gFID, yielding Pearson and Spearman coefficients of approximately 0.85—the first such metric to do so. The study further reveals that rFID and iFID capture distinct aspects of generation quality: rFID reflects characteristics of the early diffusion process, while iFID aligns with later stages. Through analysis grounded in generalization and hallucination, the authors elucidate the underlying mechanisms, demonstrating that iFID substantially outperforms conventional rFID as an evaluation metric.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Natural Language Processing: Generation
• Machine Learning: Deep Generative Models & Autoencoders

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• User Modeling, Personalization and Recommendation: Metrics for user behavior and evaluating success
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

It is well known that the reconstruction FID (rFID) of a VAE is poorly correlated with the generation FID (gFID) of a latent diffusion model. We propose interpolated FID (iFID), a simple variant of rFID that exhibits a strong correlation with gFID. Specifically, for each element in the dataset, we retrieve its nearest neighbor (NN) in the latent space and interpolate their latent representations. We then decode the interpolated latent and compute the FID between the decoded samples and the original dataset. Additionally, we refine the claim that rFID correlates poorly with gFID, by showing that rFID correlates with sample quality in the diffusion refinement phase, whereas iFID correlates with sample quality in the diffusion navigation phase. Furthermore, we provide an explanation for why iFID correlates well with gFID, and why reconstruction metrics are negatively correlated with gFID, by connecting to results in the diffusion generalization and hallucination. Empirically, iFID is the first metric to demonstrate a strong correlation with diffusion gFID, achieving Pearson linear and Spearman rank correlations approximately 0.85. The source code is provided in https://github.com/tongdaxu/Making-rFID-Predictive-of-Diffusion-gFID.

Problem

• reconstruction FID
• generation FID
• latent diffusion model
• sample quality prediction
• metric correlation

Innovation

• interpolated FID
• latent diffusion models
• reconstruction metrics
• FID correlation
• diffusion generation quality