HaloGS: Loose Coupling of Compact Geometry and Gaussian Splats for 3D Scenes

2025-05-26

AI Summary

Existing high-fidelity 3D reconstruction methods suffer from strong coupling between geometry and appearance modeling, hindering simultaneous efficiency and accuracy. This paper introduces HaloGS, the first framework to decouple geometry and appearance via a dual-representation architecture: lightweight explicit triangular meshes encode coarse-grained geometry, while differentiable Gaussian ellipsoids implicitly model fine-grained appearance details. This design overcomes the inherent accuracy–efficiency trade-off inherent in monolithic voxel-, point-, or Gaussian-based representations. By jointly optimizing geometric constraints and differentiable rendering losses, HaloGS achieves compact geometric representations (<10 MB) and state-of-the-art rendering quality—improving PSNR by 1.2 dB and SSIM by 0.02 across multiple benchmark datasets. The method significantly enhances structural robustness and detail fidelity in complex indoor and outdoor scenes.

Technology Category

• Computer Vision: 3D Computer Vision
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning
• Machine Learning: Representation Learning

Application Category

• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Security and Privacy: Data transparency and provenance
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

High fidelity 3D reconstruction and rendering hinge on capturing precise geometry while preserving photo realistic detail. Most existing methods either fuse these goals into a single cumbersome model or adopt hybrid schemes whose uniform primitives lead to a trade off between efficiency and fidelity. In this paper, we introduce HaloGS, a dual representation that loosely couples coarse triangles for geometry with Gaussian primitives for appearance, motivated by the lightweight classic geometry representations and their proven efficiency in real world applications. Our design yields a compact yet expressive model capable of photo realistic rendering across both indoor and outdoor environments, seamlessly adapting to varying levels of scene complexity. Experiments on multiple benchmark datasets demonstrate that our method yields both compact, accurate geometry and high fidelity renderings, especially in challenging scenarios where robust geometric structure make a clear difference.

Problem

• Balancing 3D reconstruction accuracy and photo-realistic detail
• Resolving efficiency-fidelity trade-offs in hybrid 3D representations
• Adapting to diverse scene complexities with compact geometry

Innovation

• Loose coupling of triangles and Gaussian splats
• Compact geometry with high fidelity rendering
• Adapts to varying scene complexity