3DGS-VBench: A Comprehensive Video Quality Evaluation Benchmark for 3DGS Compression

2025-08-09

AI Summary

To address the lack of systematic visual quality assessment for generative compression of 3D Gaussian Splatting (3DGS), this paper introduces the first dedicated video quality assessment (VQA) benchmark for 3DGS compression. Methodologically, we generate 660 compressed models and corresponding rendered videos across 11 diverse scenes using six state-of-the-art compression algorithms, and conduct a large-scale subjective study with 50 participants. Mean Opinion Score (MOS) analysis and outlier rejection ensure data reliability. Our contributions are threefold: (1) the first open-source, subjectively annotated 3DGS compression quality dataset; (2) comprehensive evaluation of six compression methods and fifteen VQA metrics; and (3) identification of 3DGS-specific distortion patterns—such as splat coherence loss and depth discontinuity artifacts—to guide the development of specialized VQA models. This work fills a critical gap in evaluating generative neural rendering compression quality.

Technology Category

• Computer Vision: 3D Computer Vision
• Knowledge Representation and Reasoning: Qualitative Reasoning
• Data Mining & Knowledge Management: Data Compression

Application Category

• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Web Mining and Content Analysis: Web data quality in the era of algorithmically-generated content

Abstract

3D Gaussian Splatting (3DGS) enables real-time novel view synthesis with high visual fidelity, but its substantial storage requirements hinder practical deployment, prompting state-of-the-art (SOTA) 3DGS methods to incorporate compression modules. However, these 3DGS generative compression techniques introduce unique distortions lacking systematic quality assessment research. To this end, we establish 3DGS-VBench, a large-scale Video Quality Assessment (VQA) Dataset and Benchmark with 660 compressed 3DGS models and video sequences generated from 11 scenes across 6 SOTA 3DGS compression algorithms with systematically designed parameter levels. With annotations from 50 participants, we obtained MOS scores with outlier removal and validated dataset reliability. We benchmark 6 3DGS compression algorithms on storage efficiency and visual quality, and evaluate 15 quality assessment metrics across multiple paradigms. Our work enables specialized VQA model training for 3DGS, serving as a catalyst for compression and quality assessment research. The dataset is available at https://github.com/YukeXing/3DGS-VBench.

Problem

• Assessing quality distortions in compressed 3DGS videos
• Benchmarking storage and visual quality of 3DGS compression
• Evaluating quality metrics for 3DGS compression techniques

Innovation

• Large-scale VQA dataset for 3DGS compression
• Benchmarking 6 SOTA 3DGS compression algorithms
• Evaluating 15 quality assessment metrics systematically