Scaling Point-based Differentiable Rendering for Large-scale Reconstruction

2025-12-22

AI Summary

Point-based differentiable rendering (PBDR) faces critical bottlenecks—including tight system coupling, poor data locality, and high communication overhead—in high-resolution and large-scale scenarios. To address these, this paper introduces Gaian, a general-purpose distributed training system for PBDR. Its core innovation is the first decoupled system architecture specifically designed for PBDR, which abstracts diverse PBDR algorithms into a unified API. Gaian explicitly models fine-grained data access patterns to enable locality-aware scheduling, communication compression, and computation-communication overlap. Evaluated across six datasets and up to 128 GPUs, Gaian reduces total communication volume by up to 91%, improves training throughput by 1.50×–3.71×, and seamlessly supports four mainstream PBDR methods. These results demonstrate substantial improvements in scalability and efficiency over existing systems.

Technology Category

• Computer Vision: Generative Adversarial Networks (GANs) for Vision
• Machine Learning: Distributed Machine Learning & Federated Learning
• Data Mining & Knowledge Management: Scalability, Parallel & Distributed Systems

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Decentralized Web and Fediverse systems
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization

Abstract

Point-based Differentiable Rendering (PBDR) enables high-fidelity 3D scene reconstruction, but scaling PBDR to high-resolution and large scenes requires efficient distributed training systems. Existing systems are tightly coupled to a specific PBDR method. And they suffer from severe communication overhead due to poor data locality. In this paper, we present Gaian, a general distributed training system for PBDR. Gaian provides a unified API expressive enough to support existing PBDR methods, while exposing rich data-access information, which Gaian leverages to optimize locality and reduce communication. We evaluated Gaian by implementing 4 PBDR algorithms. Our implementations achieve high performance and resource efficiency: across six datasets and up to 128 GPUs, it reduces communication by up to 91% and improves training throughput by 1.50x-3.71x.

Problem

• Scaling point-based differentiable rendering for large scenes
• Reducing communication overhead in distributed PBDR training
• Providing a general system for multiple PBDR algorithms

Innovation

• Gaian provides a unified API for PBDR methods
• Gaian optimizes data locality to reduce communication overhead
• Gaian improves training throughput across multiple GPUs and datasets