From Fields to Splats: A Cross-Domain Survey of Real-Time Neural Scene Representations

2025-09-27

AI Summary

This paper investigates why 3D Gaussian Splatting (3DGS) is progressively supplanting Neural Radiance Fields (NeRF) as the dominant paradigm for real-time neural scene representation. Methodologically, it conducts cross-domain comparative analysis—spanning SLAM, telepresence, robotic manipulation, and 3D content generation—and proposes a unified framework to characterize 3DGS’s synergistic advantages in rasterization compatibility, geometric fidelity, and optimization efficiency. It identifies common evolutionary patterns in multimodal input adaptation and domain-specific constraints. Empirically, 3DGS demonstrates superior rendering quality, faster convergence, and enhanced system integrability over NeRF. The study elucidates the intrinsic mechanisms underlying this paradigm shift while highlighting current limitations—particularly in dynamic scene modeling and sparse-view generalization—and proposes future directions including scalable architectures and implicit-explicit hybrid representations. Collectively, it provides theoretical foundations and practical guidance for advancing neural rendering toward real-time deployment and real-world applicability.

Technology Category

• Computer Vision: 3D Computer Vision
• Natural Language Processing: Language Grounding & Multi-modal NLP
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• User Modeling, Personalization and Recommendation: Practical large-scale studies of user experience

Abstract

Neural scene representations such as Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3DGS) have transformed how 3D environments are modeled, rendered, and interpreted. NeRF introduced view-consistent photorealism via volumetric rendering; 3DGS has rapidly emerged as an explicit, efficient alternative that supports high-quality rendering, faster optimization, and integration into hybrid pipelines for enhanced photorealism and task-driven scene understanding. This survey examines how 3DGS is being adopted across SLAM, telepresence and teleoperation, robotic manipulation, and 3D content generation. Despite their differences, these domains share common goals: photorealistic rendering, meaningful 3D structure, and accurate downstream tasks. We organize the review around unified research questions that explain why 3DGS is increasingly displacing NeRF-based approaches: What technical advantages drive its adoption? How does it adapt to different input modalities and domain-specific constraints? What limitations remain? By systematically comparing domain-specific pipelines, we show that 3DGS balances photorealism, geometric fidelity, and computational efficiency. The survey offers a roadmap for leveraging neural rendering not only for image synthesis but also for perception, interaction, and content creation across real and virtual environments.

Problem

• Comparing 3D Gaussian Splatting advantages over Neural Radiance Fields
• Analyzing cross-domain adaptation for robotics and content generation
• Identifying limitations in neural rendering for perception and interaction

Innovation

• 3D Gaussian Splatting enables efficient high-quality rendering
• It adapts to various input modalities and constraints
• Balances photorealism, geometry, and computational efficiency