Quality assessment of 3D human animation: Subjective and objective evaluation

2025-05-29

AI Summary

Non-parametric 3D virtual human animation lacks effective, standardized quality assessment methods. Method: We propose the first data-driven, end-to-end quality assessment framework, built upon a novel benchmark dataset containing user-provided subjective realism scores for virtual human animations. Our approach performs multimodal feature engineering by jointly modeling kinematic sequences and visual features, and comparatively evaluates linear regression against deep learning models. Contribution/Results: This work introduces (i) the first dedicated evaluation paradigm for non-parametric virtual human animation; (ii) the first publicly available benchmark with human-annotated subjective quality labels; and (iii) a linear regression model achieving a Pearson correlation coefficient of 0.90 with human judgments—significantly outperforming existing deep learning baselines. The framework balances high predictive performance with model interpretability, establishing a new standard for objective quality assessment in virtual human animation.

Technology Category

• Humans and AI: Game Design — Virtual Humans, NPCs and Autonomous Characters
• Computer Vision: 3D Computer Vision
• Knowledge Representation and Reasoning: Qualitative Reasoning

Application Category

• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• User Modeling, Personalization and Recommendation: Metrics for user behavior and evaluating success
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

Virtual human animations have a wide range of applications in virtual and augmented reality. While automatic generation methods of animated virtual humans have been developed, assessing their quality remains challenging. Recently, approaches introducing task-oriented evaluation metrics have been proposed, leveraging neural network training. However, quality assessment measures for animated virtual humans that are not generated with parametric body models have yet to be developed. In this context, we introduce a first such quality assessment measure leveraging a novel data-driven framework. First, we generate a dataset of virtual human animations together with their corresponding subjective realism evaluation scores collected with a user study. Second, we use the resulting dataset to learn predicting perceptual evaluation scores. Results indicate that training a linear regressor on our dataset results in a correlation of 90%, which outperforms a state of the art deep learning baseline.

Problem

• Assessing quality of non-parametric 3D human animations
• Developing data-driven metrics for animation realism
• Predicting perceptual scores via subjective dataset training

Innovation

• Data-driven framework for quality assessment
• Subjective realism scores via user study
• Linear regressor outperforms deep learning