ReLaGS: Relational Language Gaussian Splatting

2026-03-18

AI Summary

Existing 3D perception methods often rely on object-centric modeling or extensive scene-specific training, hindering unified and efficient open-vocabulary reasoning. This work proposes a training-free, unified framework that constructs a hierarchical 3D scene representation by distilling language-aligned Gaussian splats, refines geometry through Gaussian pruning, and aggregates multi-view 2D features via language-guided alignment to produce precise 3D object embeddings. Building upon this representation, the method constructs an open-vocabulary 3D semantic scene graph that jointly models hierarchical semantics and intra- and inter-object relationships, enabling unified reasoning across segmentation, retrieval, and relational understanding. Experiments demonstrate that the approach is both efficient and scalable across multiple tasks.

Technology Category

• Computer Vision: 3D Computer Vision
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning
• Machine Learning: Representation Learning

Application Category

• Semantics and Knowledge: Scalable techniques for the creation, curation, publication, maintenance, and consumption of large, Web-based, structured, reusable, knowledge graphs and ontologies
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Search and Retrieval-Augmented AI: Web query analysis, representation and understanding

Abstract

Achieving unified 3D perception and reasoning across tasks such as segmentation, retrieval, and relation understanding remains challenging, as existing methods are either object-centric or rely on costly training for inter-object reasoning. We present a novel framework that constructs a hierarchical language-distilled Gaussian scene and its 3D semantic scene graph without scene-specific training. A Gaussian pruning mechanism refines scene geometry, while a robust multi-view language alignment strategy aggregates noisy 2D features into accurate 3D object embeddings. On top of this hierarchy, we build an open-vocabulary 3D scene graph with Vision Language derived annotations and Graph Neural Network-based relational reasoning. Our approach enables efficient and scalable open-vocabulary 3D reasoning by jointly modeling hierarchical semantics and inter/intra-object relationships, validated across tasks including open-vocabulary segmentation, scene graph generation, and relation-guided retrieval. Project page: https://dfki-av.github.io/ReLaGS/

Problem

• 3D perception
• relational reasoning
• open-vocabulary
• scene graph
• unified reasoning

Innovation

• Gaussian Splatting
• Open-vocabulary 3D Reasoning
• Scene Graph Generation
• Multi-view Language Alignment
• Hierarchical Semantic Modeling