Design Space and Implementation of RAG-Based Avatars for Virtual Archaeology

2026-03-24

AI Summary

This study addresses the challenge of providing immersive, on-demand access to digital cultural heritage information in virtual archaeology. To this end, we present a Retrieval-Augmented Generation (RAG)-powered conversational avatar system integrated within a virtual reality (VR) environment, enabling users to obtain academically grounded, structured knowledge in real time while exploring digital twin reconstructions such as the 4th-century Mausoleum of Maxentius. We introduce the first design space for RAG-driven avatars tailored to virtual archaeology, encompassing application scenarios, character personas, and multimodal interaction modalities. Empirical evaluation demonstrates that our approach significantly reduces users’ perceived cognitive load while enhancing thematic engagement, thereby establishing the feasibility and innovative potential of RAG-based conversational agents in delivering immersive digital heritage experiences.

Technology Category

• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval
• Humans and AI: Game Design — Virtual Humans, NPCs and Autonomous Characters
• Cognitive Modeling & Cognitive Systems: Agent Architectures

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems

Abstract

Immersive technologies, such as virtual and augmented reality, are transforming digital heritage by enabling users to explore and interact with culturally significant sites. It is now possible to view and augment digital twins, or digitally reconstructed versions of them, and to enable access to previously unreachable locations for a broader audience. Here, we investigate retrieval-augmented generation (RAG)-based avatars as an interface for accessing further information about digital cultural heritage objects while immersed in dedicated virtual environments. We present a requirement design space that spans the application realm, avatar personality, and I/O modalities. We instantiate it with a RAG system coupled to a conversational avatar in a virtual reality (VR) environment, using the Maxentius mausoleum from the 4th century AD as a case study, through which users gain access to curated on-demand information of the digitised heritage object. Our workflow utilises scholarly texts and enriches them with metadata. We evaluate various RAG configurations in terms of answer quality on a small expert-crafted question-answer set, as well as the perceived workload of users of a VR setup using such a RAG avatar. We demonstrate evidence that users perceive the overall workload for interacting with such an avatar as below average and that such avatars help to gain topical engagement. Overall, our work demonstrates how to utilise RAG-driven VR avatars for archaeological purposes and provides evidence that they can offer a pathway for immersive, AI-enhanced digital heritage applications.

Problem

• virtual archaeology
• digital heritage
• retrieval-augmented generation
• immersive technologies
• conversational avatars

Innovation

• Retrieval-Augmented Generation (RAG)
• Virtual Reality Avatars
• Digital Heritage
• Immersive Archaeology
• Conversational AI