3DGS$^3$: Joint Super Sampling and Frame Interpolation for Real-Time Large-Scale 3DGS Rendering

2026-05-12

AI Summary

This work addresses the computational bottleneck of 3D Gaussian Splatting (3DGS) in ultra-dense scenes and high-resolution settings, which hinders its ability to meet low-latency rendering requirements. To overcome this limitation without altering the original 3DGS pipeline, the authors propose 3DGS³, a unified post-rendering framework that jointly enables gradient-aware super-resolution and lightweight temporal frame interpolation. Leveraging the continuous differentiability of 3DGS, the method employs a GRU-based Gradient-Aware Super-Sampling (GASS) module to extract spatial gradients and integrates a U-Net–inspired Lightweight Temporal Frame Interpolation (LTFI) module to fuse temporal information, thereby simultaneously enhancing both resolution and frame rate. Experiments demonstrate that 3DGS³ significantly outperforms existing approaches on public benchmarks, achieving state-of-the-art performance in both rendering efficiency and visual quality.

Abstract

3D Gaussian Splatting (3DGS) enables high-quality real-time 3D rendering but faces challenges in efficiently scaling to ultra-dense scenes and high-resolution due to computational bottlenecks that limit its use in latency-sensitive applications. Instead of optimizing the splatting pipeline itself, we propose \textbf{3DGS$^3$}, a unified post-rendering framework that jointly performs super sampling and frame interpolation through differentiable processing of low-resolution outputs to achieve both high-resolution and high-frame-rate rendering. Our \textbf{Gradient\- \-Aware Super Sampling (GASS)} module leverages the continuous differentiability of 3DGS to extract image gradients that guide a GRU-based refinement network to enable high-fidelity super sampling. Furthermore, a \textbf{Lightweight Temporal Frame Interpolation (LTFI)} module based on a compact U-Net-like backbone fuses temporal and differentiable spatial cues from consecutive frames to synthesize temporally coherent intermediate frames. Experiments on public datasets demonstrate that 3DGS$^3$ achieves superior rendering efficiency and visual quality when compared with state-of-the-art methods and remains compatible with existing 3DGS acceleration techniques. The code will be publicly released upon acceptance.

Problem

• 3D Gaussian Splatting
• real-time rendering
• computational bottleneck
• high-resolution
• latency-sensitive applications

Innovation

• 3D Gaussian Splatting
• Super Sampling
• Frame Interpolation
• Differentiable Rendering
• Real-Time Rendering