Generative Models at the Frontier of Compression: A Survey on Generative Face Video Coding

2025-06-09

AI Summary

Conventional video coding standards (e.g., VVC) struggle to achieve high-fidelity facial video reconstruction at ultra-low bitrates (<0.1 bpp). Method: This work proposes Generative Facial Video Coding (GFVC), a novel end-to-end semantic compression paradigm integrating variational autoencoders (VAEs), generative adversarial networks (GANs), and diffusion models—unifying high-level semantic representation with lightweight system implementation. We introduce the first large-scale GFVC benchmark dataset annotated with subjective Mean Opinion Scores (MOS), design a rate-distortion optimization framework, and define standardized inference interfaces. Results: Experiments demonstrate that GFVC significantly outperforms VVC across 0.01–0.1 bpp in PSNR, SSIM, and MOS. Moreover, we identify fundamental performance bottlenecks of GFVC and establish empirical relationships between reconstruction quality metrics and perceptual fidelity—providing theoretical foundations and practical guidelines for industrial deployment and standardization.

Technology Category

• Computer Vision: Generative Adversarial Networks (GANs) for Vision
• Machine Learning: Deep Generative Models & Autoencoders
• Natural Language Processing: Generation

Application Category

• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

The rise of deep generative models has greatly advanced video compression, reshaping the paradigm of face video coding through their powerful capability for semantic-aware representation and lifelike synthesis. Generative Face Video Coding (GFVC) stands at the forefront of this revolution, which could characterize complex facial dynamics into compact latent codes for bitstream compactness at the encoder side and leverages powerful deep generative models to reconstruct high-fidelity face signal from the compressed latent codes at the decoder side. As such, this well-designed GFVC paradigm could enable high-fidelity face video communication at ultra-low bitrate ranges, far surpassing the capabilities of the latest Versatile Video Coding (VVC) standard. To pioneer foundational research and accelerate the evolution of GFVC, this paper presents the first comprehensive survey of GFVC technologies, systematically bridging critical gaps between theoretical innovation and industrial standardization. In particular, we first review a broad range of existing GFVC methods with different feature representations and optimization strategies, and conduct a thorough benchmarking analysis. In addition, we construct a large-scale GFVC-compressed face video database with subjective Mean Opinion Scores (MOSs) based on human perception, aiming to identify the most appropriate quality metrics tailored to GFVC. Moreover, we summarize the GFVC standardization potentials with a unified high-level syntax and develop a low-complexity GFVC system which are both expected to push forward future practical deployments and applications. Finally, we envision the potential of GFVC in industrial applications and deliberate on the current challenges and future opportunities.

Problem

• Surveying generative face video coding (GFVC) technologies for compression.
• Benchmarking GFVC methods and analyzing quality metrics.
• Exploring GFVC standardization and practical deployment potential.

Innovation

• Generative models compress face video efficiently
• Latent codes enable ultra-low bitrate communication
• Standardization and benchmarking advance GFVC adoption