SFQA: A Comprehensive Perceptual Quality Assessment Dataset for Singing Face Generation

2026-01-28

AI Summary

This work addresses the lack of a unified, high-quality evaluation benchmark for singing face generation. To this end, we present SFQA—the first perceptual quality assessment dataset specifically designed for singing face synthesis—comprising 5,184 video samples generated from 12 state-of-the-art methods across 100 identities and 36 multi-style musical pieces. Large-scale subjective evaluations were conducted to collect human quality ratings, establishing a reliable ground truth for perceptual assessment. The dataset fills a critical gap in evaluation resources for this domain, systematically revealing performance disparities among existing methods under singing-specific conditions. Furthermore, SFQA serves as a standardized platform for benchmarking current objective evaluation metrics, thereby laying a solid foundation for future algorithm development and rigorous performance evaluation in singing face generation.

Technology Category

• Natural Language Processing: Generation
• Computer Vision: Generative Adversarial Networks (GANs) for Vision
• Cognitive Modeling & Cognitive Systems: Affective Computing

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Web Mining and Content Analysis: Web data quality in the era of algorithmically-generated content

Abstract

The Talking Face Generation task has enormous potential for various applications in digital humans and agents, etc. Singing, as a common facial movement second only to talking, can be regarded as a universal language across ethnicities and cultures. However, it is often underestimated in the field due to lack of singing face datasets and the domain gap between singing and talking in rhythm and amplitude. More significantly, the quality of Singing Face Generation (SFG) often falls short and is uneven or limited by different applicable scenarios, which prompts us to propose timely and effective quality assessment methods to ensure user experience. To address existing gaps in this domain, this paper introduces a new SFG content quality assessment dataset SFQA, built using 12 representative generation methods. During the construction of the dataset, 100 photographs or portraits, as well as 36 music clips from 7 different styles, are utilized to generate 5,184 singing face videos that constitute the SFQA dataset. To further explore the quality of SFG methods, subjective quality assessment is conducted by evaluators, whose ratings reveal a significant variation in quality among different generation methods. Based on our proposed SFQA dataset, we comprehensively benchmark the current objective quality assessment algorithms.

Problem

• Singing Face Generation
• Perceptual Quality Assessment
• Talking Face Generation
• Dataset
• Digital Humans

Innovation

• Singing Face Generation
• Perceptual Quality Assessment
• SFQA Dataset
• Subjective Evaluation
• Talking Face Generation