FlexGS: Train Once, Deploy Everywhere with Many-in-One Flexible 3D Gaussian Splatting

2025-06-04

AI Summary

To address the memory constraints and inflexible adaptability of 3D Gaussian Splatting (3DGS) on heterogeneous devices, this paper proposes an elastic inference paradigm that enables dynamic model compression—without fine-tuning—to meet arbitrary target memory budgets after a single training phase. The core components are a learnable Gaussian selector and a parameter-adaptive transformation module, jointly optimized to proportionally prune Gaussians and recalibrate their spatial distribution. The method is compatible with mainstream datasets including ZipNeRF, MipNeRF, and Tanks & Temples. Across diverse scenes, it reduces GPU memory consumption by up to 75% while preserving rendering quality close to that of the full model. It achieves low deployment latency and strong generalization across unseen scenes. By significantly enhancing practicality and deployment flexibility, our approach advances the applicability of 3DGS on resource-constrained edge devices.

Technology Category

• Machine Learning: Learning on the Edge & Model Compression
• Computer Vision: 3D Computer Vision
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

3D Gaussian splatting (3DGS) has enabled various applications in 3D scene representation and novel view synthesis due to its efficient rendering capabilities. However, 3DGS demands relatively significant GPU memory, limiting its use on devices with restricted computational resources. Previous approaches have focused on pruning less important Gaussians, effectively compressing 3DGS but often requiring a fine-tuning stage and lacking adaptability for the specific memory needs of different devices. In this work, we present an elastic inference method for 3DGS. Given an input for the desired model size, our method selects and transforms a subset of Gaussians, achieving substantial rendering performance without additional fine-tuning. We introduce a tiny learnable module that controls Gaussian selection based on the input percentage, along with a transformation module that adjusts the selected Gaussians to complement the performance of the reduced model. Comprehensive experiments on ZipNeRF, MipNeRF and Tanks&Temples scenes demonstrate the effectiveness of our approach. Code is available at https://flexgs.github.io.

Problem

• Reducing GPU memory usage for 3D Gaussian splatting
• Enabling flexible deployment across varied computational devices
• Eliminating need for fine-tuning after model compression

Innovation

• Elastic inference method for 3DGS
• Tiny learnable module controls selection
• Transformation module adjusts selected Gaussians