Gaussian RBFNet: Gaussian Radial Basis Functions for Fast and Accurate Representation and Reconstruction of Neural Fields

2025-03-09

AI Summary

Neural fields suffer from slow training, inefficient inference, and high memory consumption when modeling 2D images, 3D geometry, and 5D radiance fields. To address these challenges, this paper proposes Gaussian RBFNet—a novel neural field architecture that replaces conventional multi-layer perceptrons (MLPs) with a single-layer network using Gaussian radial basis function (RBF) kernels. Integrated with a low-resolution feature grid and parallelized computation, Gaussian RBFNet enables end-to-end lightweight modeling. It achieves state-of-the-art reconstruction accuracy while dramatically improving efficiency: 3D geometry reconstruction trains in under 15 seconds; novel-view synthesis trains in under 15 minutes; and real-time rendering attains over 60 FPS. The core contribution lies in substituting deep MLPs with a single-layer Gaussian RBF representation, achieving an unprecedented balance among expressive power, computational speed, and memory efficiency.

Technology Category

• Computer Vision: Generative Adversarial Networks (GANs) for Vision
• Machine Learning: Learning with Manifolds
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Architectures and workflows that use LLMs for crowd work

Abstract

Neural fields such as DeepSDF and Neural Radiance Fields have recently revolutionized novel-view synthesis and 3D reconstruction from RGB images and videos. However, achieving high-quality representation, reconstruction, and rendering requires deep neural networks, which are slow to train and evaluate. Although several acceleration techniques have been proposed, they often trade off speed for memory. Gaussian splatting-based methods, on the other hand, accelerate the rendering time but remain costly in terms of training speed and memory needed to store the parameters of a large number of Gaussians. In this paper, we introduce a novel neural representation that is fast, both at training and inference times, and lightweight. Our key observation is that the neurons used in traditional MLPs perform simple computations (a dot product followed by ReLU activation) and thus one needs to use either wide and deep MLPs or high-resolution and high-dimensional feature grids to parameterize complex nonlinear functions. We show in this paper that by replacing traditional neurons with Radial Basis Function (RBF) kernels, one can achieve highly accurate representation of 2D (RGB images), 3D (geometry), and 5D (radiance fields) signals with just a single layer of such neurons. The representation is highly parallelizable, operates on low-resolution feature grids, and is compact and memory-efficient. We demonstrate that the proposed novel representation can be trained for 3D geometry representation in less than 15 seconds and for novel view synthesis in less than 15 mins. At runtime, it can synthesize novel views at more than 60 fps without sacrificing quality.

Problem

• Achieves fast, accurate neural field representation and reconstruction.
• Reduces training and inference time with lightweight, memory-efficient design.
• Enables high-quality 3D geometry and novel-view synthesis at 60 fps.

Innovation

• Uses Gaussian Radial Basis Function kernels
• Replaces traditional MLP neurons for efficiency
• Achieves fast training and high-quality rendering