CreepyCoCreator? Investigating AI Representation Modes for 3D Object Co-Creation in Virtual Reality

2025-02-05

AI Summary

Current VR world-building systems lack a principled understanding of human-AI co-creation mechanisms, resulting in misalignment between user intent and AI-generated content. This paper investigates the interactive process of collaborative 3D object creation between users and generative AI in VR, introducing— for the first time—the systematic examination of three embodied intent representation modalities: avatar-guided interaction, progressive/immediate visualization, and pre-modification region highlighting. Through a Wizard-of-Oz study and mixed-methods evaluation, we demonstrate that their synergistic integration significantly enhances users’ sense of control, trust, and creative satisfaction. Building on these findings, we propose an AI explainability design framework grounded in human collaboration metaphors. This framework provides empirically validated design guidelines and theoretical foundations for next-generation VR world-building systems.

Technology Category

• Humans and AI: Game Design — Virtual Humans, NPCs and Autonomous Characters
• Intelligent Robots: Embodied AI
• Cognitive Modeling & Cognitive Systems: Computational Creativity

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI

Abstract

Generative AI in Virtual Reality enables users to create detailed immersive worlds with a rich variety However current worldbuilding systems often lack an understanding of the fundamental aspects of human-AI cocreation resulting in a disconnect between user intent and AIgenerated content This paper investigates the co-creative process between users and an object-generating AI in Virtual Reality Through a WizardofOz study we explore how AI can represent its intent to users when customizing objects Inspired by human-to-human collaboration we focus on three representation modes the presence of an embodied avatar whether the AIs contributions are visualized immediately or incrementally and whether the areas modified are highlighted in advance The findings provide insights into how these factors affect user perception and interaction with object-generating AI in Virtual Reality The results offer design implications for co-creative worldbuilding systems aiming to foster more effective and satisfying collaborations between humans and AI in Virtual Reality.

Problem

• AI representation in VR co-creation
• User intent and AI content disconnect
• Effective human-AI collaboration modes

Innovation

• AI representation modes
• Wizard-of-Oz study
• co-creative process