Visionary: The World Model Carrier Built on WebGPU-Powered Gaussian Splatting Platform
🤖 AI Summary
Current Web-based solutions for 3D Gaussian Splatting (3DGS) and neural rendering are fragmented and deployment-heavy, lacking support for dynamic content and generative modeling. This paper introduces the first lightweight, WebGPU-native platform enabling real-time neural rendering and dynamic generative 3D modeling directly in browsers. Our approach unifies reconstruction and generation within a single framework. Key contributions include: (1) a standardized Gaussian generator protocol supporting per-frame ONNX inference and plugin-based Gaussian distribution updates; (2) the first in-browser integration of 3DGS reconstruction with generative paradigms—fully compatible with MLP-based 3DGS, 4DGS, neural avatars, and stylized post-processing; and (3) a TypeScript-wrapped three.js plugin library enabling low-barrier deployment, reproducibility, and cross-method evaluation. Experiments demonstrate significantly higher rendering efficiency than existing Web viewers under identical asset conditions, accelerating the practical adoption of the 3DGS family of methods.
Application Category
📝 Abstract
Neural rendering, particularly 3D Gaussian Splatting (3DGS), has evolved rapidly and become a key component for building world models. However, existing viewer solutions remain fragmented, heavy, or constrained by legacy pipelines, resulting in high deployment friction and limited support for dynamic content and generative models. In this work, we present Visionary, an open, web-native platform for real-time various Gaussian Splatting and meshes rendering. Built on an efficient WebGPU renderer with per-frame ONNX inference, Visionary enables dynamic neural processing while maintaining a lightweight, "click-to-run" browser experience. It introduces a standardized Gaussian Generator contract, which not only supports standard 3DGS rendering but also allows plug-and-play algorithms to generate or update Gaussians each frame. Such inference also enables us to apply feedforward generative post-processing. The platform further offers a plug in three.js library with a concise TypeScript API for seamless integration into existing web applications. Experiments show that, under identical 3DGS assets, Visionary achieves superior rendering efficiency compared to current Web viewers due to GPU-based primitive sorting. It already supports multiple variants, including MLP-based 3DGS, 4DGS, neural avatars, and style transformation or enhancement networks. By unifying inference and rendering directly in the browser, Visionary significantly lowers the barrier to reproduction, comparison, and deployment of 3DGS-family methods, serving as a unified World Model Carrier for both reconstructive and generative paradigms.
Problem
Research questions and friction points this paper is trying to address.
Fragmented viewer solutions hinder 3D Gaussian Splatting deployment and dynamic content support
Existing pipelines are heavy and constrained, limiting real-time neural rendering in browsers
Lack of a unified, open web platform for integrating and deploying 3DGS-family methods
Innovation
Methods, ideas, or system contributions that make the work stand out.
WebGPU-based real-time Gaussian Splatting rendering platform
Per-frame ONNX inference for dynamic neural content processing
Standardized plug-and-play Gaussian Generator contract system
🔎 Similar Papers
No similar papers found.
