AdvFD: Boosting Visual Generation via Adversarial Fr'echet Distance Loss

2026-08-11

AI Summary

This work addresses the issue of “Fréchet hacking” in generative model optimization, where Fréchet distance losses based on static pretrained feature spaces yield deceptively high scores despite degraded visual quality and poor cross-feature alignment. To mitigate this, the authors propose the adversarial Fréchet distance (AdvFD) loss, which introduces adversarial learning into Fréchet distance optimization for the first time. AdvFD constructs a learnable, adaptive feature space that dynamically enhances distribution discrepancy measurement and incorporates a real-feature whitening mechanism to suppress feature amplification and stabilize training. The method consistently improves both visual fidelity and distribution alignment in single-step generator post-training across various model scales and backbone architectures, including JiT and pMF.

Abstract

Fréchet distance has recently emerged as an effective distribution-level objective for generator post-training, complementing the conventional sample-level diffusion and flow-matching losses. However, directly optimizing Fréchet objectives can cause Fréchet hacking. The target metrics keep improving, but visual quality and Fréchet alignment in other feature spaces may stagnate or deteriorate. We attribute this failure to the static pretrained feature spaces used by existing Fréchet losses. These feature spaces provide incomplete and fixed views of the differences between real and generated distributions. To address this limitation, we propose Adversarial Fréchet Distance (AdvFD), which complements the static representation targets in FD-Loss with a calibrated adversarially learned representation. AdvFD augments the original static Fréchet objective with a learnable representation that adversarially maximizes the Fréchet discrepancy between real and generated samples, while the generator minimizes the same discrepancy in the resulting adaptive feature space. To prevent the adversarial representation from trivially increasing the objective through feature amplification, we further introduce real-feature whitening, which normalizes its scale and covariance geometry and stabilizes the min--max optimization. Extensive experiments show that AdvFD consistently improves one-step generator post-training across both JiT and pMF backbones and across different model scales.

Problem

• Fréchet distance
• generator post-training
• Fréchet hacking
• static feature spaces
• visual quality

Innovation

• Adversarial Fréchet Distance
• generator post-training
• feature space adaptation
• real-feature whitening
• distribution alignment