Gesture Evaluation in Virtual Reality

2025-09-16

AI Summary

Prior work lacks systematic comparison of how immersive virtual reality (VR) versus conventional 2D video affects subjective perception of AI-generated gestures—particularly whether VR enhances the validity of nonverbal behavior evaluation. Method: We conducted a controlled user study using three state-of-the-art gesture generation models from the GENEA Challenge 2023, evaluated under motion-capture-driven VR and temporally synchronized 2D video conditions; participants rated gestures on naturalness and related dimensions. Contribution/Results: VR yielded significantly higher naturalness ratings (p < 0.05), especially amplifying the perceptual advantage of motion-capture-based models. Crucially, model rankings remained highly consistent across modalities (Spearman ρ = 0.92), demonstrating VR’s dual sensitivity and stability. This work establishes VR as a valid, ecologically grounded paradigm for evaluating embodied AI nonverbal behavior—providing methodological foundations for future assessments of human-AI interaction.

Technology Category

• Humans and AI: Game Design — Virtual Humans, NPCs and Autonomous Characters
• Computer Vision: Biometrics, Face, Gesture & Pose
• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• User Modeling, Personalization and Recommendation: Metrics for user behavior and evaluating success
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI

Abstract

Gestures are central to human communication, enriching interactions through non-verbal expression. Virtual avatars increasingly use AI-generated gestures to enhance life-likeness, yet evaluations have largely been confined to 2D. Virtual Reality (VR) provides an immersive alternative that may affect how gestures are perceived. This paper presents a comparative evaluation of computer-generated gestures in VR and 2D, examining three models from the 2023 GENEA Challenge. Results show that gestures viewed in VR were rated slightly higher on average, with the strongest effect observed for motion-capture "true movement." While model rankings remained consistent across settings, VR influenced participants' overall perception and offered unique benefits over traditional 2D evaluation.

Problem

• Evaluating AI-generated gestures in VR versus 2D
• Comparing perception of computer-generated gestures across mediums
• Assessing VR's impact on gesture evaluation methodology

Innovation

• VR-based gesture evaluation method
• Comparative analysis of 3 AI models
• Motion-capture true movement enhancement