HSG: Hyperbolic Scene Graph

2026-04-19

AI Summary

Existing scene graph methods struggle to explicitly model the hierarchical entailment relations between locations and objects in Euclidean space, leading to insufficient structural consistency. This work proposes the first approach that incorporates hyperbolic geometry into scene graph representation learning, leveraging its innate capacity for encoding hierarchical structures. By integrating contrastive learning with attention mechanisms, the method enables more structured visual understanding. The resulting embeddings exhibit significantly improved hierarchical organization and semantic coherence, achieving a Graph IoU score of 33.51—representing an absolute gain of 8.14 over the strongest baseline—while maintaining competitive performance in retrieval tasks.

Technology Category

• Computer Vision: Representation Learning for Vision
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning
• Machine Learning: Representation Learning

Application Category

• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web query analysis, representation and understanding

Abstract

Scene graph representations enable structured visual understanding by modeling objects and their relationships, and have been widely used for multiview and 3D scene reasoning. Existing methods such as MSG learn scene graph embeddings in Euclidean space using contrastive learning and attention based association. However, Euclidean geometry does not explicitly capture hierarchical entailment relationships between places and objects, limiting the structural consistency of learned representations. To address this, we propose Hyperbolic Scene Graph (HSG), which learns scene graph embeddings in hyperbolic space where hierarchical relationships are naturally encoded through geometric distance. Our results show that HSG improves hierarchical structure quality while maintaining strong retrieval performance. The largest gains are observed in graph level metrics: HSG achieves a PP IoU of 33.17 and the highest Graph IoU of 33.51, outperforming the best AoMSG variant (25.37) by 8.14, highlighting the effectiveness of hyperbolic representation learning for scene graph modeling. Code: https://github.com/AIGeeksGroup/HSG.

Problem

• scene graph
• hierarchical relationships
• hyperbolic space
• Euclidean geometry
• structured visual understanding

Innovation

• hyperbolic space
• scene graph
• hierarchical representation
• geometric embedding
• visual reasoning