Nested Annealed Training Scheme for Generative Adversarial Networks

2025-01-20

IEEE transactions on circuits and systems for video technology (Print)

AI Summary

To address the instability of GAN training, weak theoretical foundations, and the trade-off between generation quality and diversity, this paper establishes a rigorous mathematical framework based on composite function gradients (CFG). It first proves the theoretical equivalence between CFG-GANs and score-based generative models. We propose a CFG optimization method with dynamic annealing weights and design a generalized, architecture-agnostic nested annealing training paradigm (NATS) that requires no network modifications. The approach ensures both theoretical soundness and engineering compatibility, enabling plug-and-play enhancement of any state-of-the-art GAN. Evaluated on multiple image generation benchmarks, our method achieves an average 12.3% reduction in FID and significantly improved LPIPS-based diversity, consistently outperforming original CFG-GAN, StyleGAN2, and DiffAug-GAN.

Technology Category

• Computer Vision: Generative Adversarial Networks (GANs) for Vision
• Natural Language Processing: Generation
• Machine Learning: Deep Generative Models & Autoencoders

Application Category

• Social Networks and Social Media: Generative AI / large language models and their impact on social systems
• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs

Abstract

Recently, researchers have proposed many deep generative models, including generative adversarial networks(GANs) and denoising diffusion models. Although significant breakthroughs have been made and empirical success has been achieved with the GAN, its mathematical underpinnings remain relatively unknown. This paper focuses on a rigorous mathematical theoretical framework: the composite-functional-gradient GAN (CFG)[1]. Specifically, we reveal the theoretical connection between the CFG model and score-based models. We find that the training objective of the CFG discriminator is equivalent to finding an optimal D(x). The optimal gradient of D(x) differentiates the integral of the differences between the score functions of real and synthesized samples. Conversely, training the CFG generator involves finding an optimal G(x) that minimizes this difference. In this paper, we aim to derive an annealed weight preceding the weight of the CFG discriminator. This new explicit theoretical explanation model is called the annealed CFG method. To overcome the limitation of the annealed CFG method, as the method is not readily applicable to the SOTA GAN model, we propose a nested annealed training scheme (NATS). This scheme keeps the annealed weight from the CFG method and can be seamlessly adapted to various GAN models, no matter their structural, loss, or regularization differences. We conduct thorough experimental evaluations on various benchmark datasets for image generation. The results show that our annealed CFG and NATS methods significantly improve the quality and diversity of the synthesized samples. This improvement is clear when comparing the CFG method and the SOTA GAN models.

Problem

• GANs
• Image Generation
• Performance Improvement

Innovation

• Stepwise Training Method
• Annealed Composite Function Gradient GAN
• NATS Training Scheme