Representation Fréchet Loss for Visual Generation

2026-04-30

AI Summary

This work addresses the challenges of directly optimizing the Fréchet Distance (FD) in generative model training and the inconsistency between Fréchet Inception Distance (FID) and visual quality when relying solely on Inception features. The authors propose a differentiable optimization of FD in representation space, decoupling the large-scale samples required for FD estimation from minibatch-based gradient computation—enabling, for the first time, end-to-end training with FD as a loss function. Their approach compresses multi-step generators into high-performance single-step models without requiring distillation or adversarial training, and introduces FDr^k, a multi-representation-space evaluation metric. On ImageNet 256×256, the single-step generator achieves an FID of 0.72, markedly improving visual fidelity while revealing representation-dependent biases inherent in conventional FID evaluation.

Abstract

We show that Fréchet Distance (FD), long considered impractical as a training objective, can in fact be effectively optimized in the representation space. Our idea is simple: decouple the population size for FD estimation (e.g., 50k) from the batch size for gradient computation (e.g., 1024). We term this approach FD-loss. Optimizing FD-loss reveals several surprising findings. First, post-training a base generator with FD-loss in different representation spaces consistently improves visual quality. Under the Inception feature space, a one-step generator achieves0.72 FID on ImageNet 256x256. Second, the same FD-loss repurposes multi-step generators into strong one-step generators without teacher distillation, adversarial training or per-sample targets. Third, FID can misrank visual quality: modern representations can yield better samples despite worse Inception FID. This motivates FDr$^k$, a multi-representation metric. We hope this work will encourage further exploration of distributional distances in diverse representation spaces as both training objectives and evaluation metrics for generative models.

Problem

• Fréchet Distance
• Visual Generation
• Representation Space
• FID
• Generative Models

Innovation

• Fréchet Distance
• representation space
• FD-loss
• generative models
• FDr^k