Radiance Fields in XR: A Survey on How Radiance Fields are Envisioned and Addressed for XR Research

2025-08-06

AI Summary

Despite breakthroughs in radiance fields (RFs) for novel view synthesis, systematic investigation of RFs in extended reality (XR) remains severely lacking. This paper presents the first interdisciplinary, XR-oriented survey of RF research, synthesizing 365 works from computer vision, computer graphics, and human–computer interaction, and rigorously analyzing 66 deployed RF+XR systems. We identify a structural gap between XR-specific requirements—such as ultra-low latency, real-time interactivity, and multimodal perception—and prevailing RF methodologies by examining research visions, implementation paradigms, and technical constraints. To bridge this gap, we propose the first structured knowledge framework for RF+XR integration, delineating five core research directions. Furthermore, we construct a comprehensive resource map that charts technological evolution and application mappings across XR domains. This work establishes both a theoretical foundation and a practical roadmap for deep RF–XR convergence.

Technology Category

• Computer Vision: Computational Photography, Image & Video Synthesis
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Knowledge Representation and Reasoning: Applications

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Web applications in cross-disciplinary domains and verticals such as mixed reality, smart cities, and digital health
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

The development of radiance fields (RF), such as 3D Gaussian Splatting (3DGS) and Neural Radiance Fields (NeRF), has revolutionized interactive photorealistic view synthesis and presents enormous opportunities for XR research and applications. However, despite the exponential growth of RF research, RF-related contributions to the XR community remain sparse. To better understand this research gap, we performed a systematic survey of current RF literature to analyze (i) how RF is envisioned for XR applications, (ii) how they have already been implemented, and (iii) the remaining research gaps. We collected 365 RF contributions related to XR from computer vision, computer graphics, robotics, multimedia, human-computer interaction, and XR communities, seeking to answer the above research questions. Among the 365 papers, we performed an analysis of 66 papers that already addressed a detailed aspect of RF research for XR. With this survey, we extended and positioned XR-specific RF research topics in the broader RF research field and provide a helpful resource for the XR community to navigate within the rapid development of RF research.

Problem

• Analyzing how radiance fields are envisioned for XR applications
• Examining current implementations of radiance fields in XR
• Identifying research gaps in radiance fields for XR

Innovation

• Systematic survey of radiance fields for XR
• Analysis of 66 XR-focused RF research papers
• Integration of RF advancements into XR applications