3DGS$^2$-TR: Scalable Second-Order Trust-Region Method for 3D Gaussian Splatting

2026-01-30

AI Summary

This work addresses the high computational and memory costs of second-order optimization in 3D Gaussian Splatting (3DGS) by proposing an efficient diagonal Hessian approximation optimizer. It introduces, for the first time in 3DGS, a matrix-free second-order method that leverages Hutchinson’s estimator to efficiently approximate the diagonal of the Hessian. Combined with a parameter-wise trust-region mechanism based on the Hellinger distance, the approach maintains O(n) complexity—comparable to Adam—while ensuring training stability. Experimental results demonstrate that, under identical initialization and without densification, the method reduces the required training iterations by 50%, incurs only a 17% higher peak GPU memory usage compared to Adam, and achieves superior reconstruction quality over existing baselines.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Machine Learning: Optimization
• Computer Vision: Learning & Optimization for CV

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

We propose 3DGS$^2$-TR,a second-order optimizer for accelerating the scene training problem in 3D Gaussian Splatting (3DGS). Unlike existing second-order approaches that rely on explicit or dense curvature representations, such as 3DGS-LM (H\"ollein et al., 2025) or 3DGS2 (Lan et al., 2025), our method approximates curvature using only the diagonal of the Hessian matrix, efficiently via Hutchinson's method. Our approach is fully matrix-free and has the same complexity as ADAM (Kingma, 2024), $O(n)$ in both computation and memory costs. To ensure stable optimization in the presence of strong nonlinearity in the 3DGS rasterization process, we introduce a parameter-wise trust-region technique based on the squared Hellinger distance, regularizing updates to Gaussian parameters. Under identical parameter initialization and without densification, 3DGS$^2$-TR is able to achieve better reconstruction quality on standard datasets, using 50% fewer training iterations compared to ADAM, while incurring less than 1GB of peak GPU memory overhead (17% more than ADAM and 85% less than 3DGS-LM), enabling scalability to very large scenes and potentially to distributed training settings.

Problem

• 3D Gaussian Splatting
• optimization
• scalability
• trust-region
• second-order method

Innovation

• second-order optimization
• 3D Gaussian Splatting
• trust-region method
• Hessian diagonal approximation
• scalable rendering