FHGS: Feature-Homogenized Gaussian Splatting

2025-05-25

AI Summary

To resolve the fundamental conflict between anisotropic Gaussian rendering and isotropic semantic feature requirements in 3D Gaussian Splatting (3DGS), this paper proposes a Feature Homogenization Fusion framework. Our method introduces a non-differentiable isotropic feature fusion mechanism to faithfully map pre-trained 2D semantic features (e.g., from SAM and CLIP) into 3D scenes; a potential-field-inspired dual-drive optimization strategy that jointly enforces global semantic alignment and local geometric consistency; and integrates sparse voxel modeling, physics-informed feature field coupling, and clustering-guided semantic supervision. While preserving real-time rendering efficiency, our approach significantly improves cross-view feature consistency. Extensive experiments demonstrate state-of-the-art performance in semantic reconstruction accuracy and interactive quality across multiple benchmarks.

Technology Category

• Computer Vision: 3D Computer Vision
• Machine Learning: Feature Construction/Reformulation
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Vertical and domain-specific search
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs

Abstract

Scene understanding based on 3D Gaussian Splatting (3DGS) has recently achieved notable advances. Although 3DGS related methods have efficient rendering capabilities, they fail to address the inherent contradiction between the anisotropic color representation of gaussian primitives and the isotropic requirements of semantic features, leading to insufficient cross-view feature consistency. To overcome the limitation, we proposes $ extit{FHGS}$ (Feature-Homogenized Gaussian Splatting), a novel 3D feature fusion framework inspired by physical models, which can achieve high-precision mapping of arbitrary 2D features from pre-trained models to 3D scenes while preserving the real-time rendering efficiency of 3DGS. Specifically, our $ extit{FHGS}$ introduces the following innovations: Firstly, a universal feature fusion architecture is proposed, enabling robust embedding of large-scale pre-trained models' semantic features (e.g., SAM, CLIP) into sparse 3D structures. Secondly, a non-differentiable feature fusion mechanism is introduced, which enables semantic features to exhibit viewpoint independent isotropic distributions. This fundamentally balances the anisotropic rendering of gaussian primitives and the isotropic expression of features; Thirdly, a dual-driven optimization strategy inspired by electric potential fields is proposed, which combines external supervision from semantic feature fields with internal primitive clustering guidance. This mechanism enables synergistic optimization of global semantic alignment and local structural consistency. More interactive results can be accessed on: https://fhgs.cuastro.org/.

Problem

• Resolves anisotropic color vs isotropic semantic feature conflict in 3DGS
• Maps 2D features to 3D scenes while maintaining real-time rendering
• Balances anisotropic rendering with isotropic feature distributions via novel fusion

Innovation

• Universal feature fusion for large-scale models
• Non-differentiable isotropic feature distribution
• Dual-driven optimization with semantic fields