Neural Spacetimes for DAG Representation Learning

2024-08-25

arXiv.org

AI Summary

This work addresses representation learning for weighted directed acyclic graphs (DAGs). We propose a neural spatiotemporal geometric modeling framework that treats nodes as events on a spacetime manifold, jointly encoding edge weights (spatial dimension) and causal order (temporal dimension). Our approach pioneers the co-modeling of a learnable neural quasimetric and a neural partial order structure; we theoretically prove it satisfies a universal embedding theorem, guaranteeing exact preservation of causal structure and logarithmic-dimensional compression. The architecture employs a tripartite network—embedding, quasimetric, and partial-order networks—operating on a product manifold, integrating fractal geometric priors with data-driven geometric adaptation. Experiments on synthetic and real-world DAGs demonstrate significantly reduced embedding distortion compared to fixed-spacetime baselines (e.g., Minkowski, de Sitter). Our method achieves subcubic parameter complexity in node count and scales linearly with DAG width.

Technology Category

• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning
• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems
• Reasoning under Uncertainty: Graphical Models

Application Category

• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Semantics and Knowledge: Methods, algorithms and applications for the development of semantic models, knowledge graphs and other forms of structured data models with machine-interpretable semantics
• Search and Retrieval-Augmented AI: Web query analysis, representation and understanding

Abstract

We propose a class of trainable deep learning-based geometries called Neural Spacetimes (NSTs), which can universally represent nodes in weighted directed acyclic graphs (DAGs) as events in a spacetime manifold. While most works in the literature focus on undirected graph representation learning or causality embedding separately, our differentiable geometry can encode both graph edge weights in its spatial dimensions and causality in the form of edge directionality in its temporal dimensions. We use a product manifold that combines a quasi-metric (for space) and a partial order (for time). NSTs are implemented as three neural networks trained in an end-to-end manner: an embedding network, which learns to optimize the location of nodes as events in the spacetime manifold, and two other networks that optimize the space and time geometries in parallel, which we call a neural (quasi-)metric and a neural partial order, respectively. The latter two networks leverage recent ideas at the intersection of fractal geometry and deep learning to shape the geometry of the representation space in a data-driven fashion, unlike other works in the literature that use fixed spacetime manifolds such as Minkowski space or De Sitter space to embed DAGs. Our main theoretical guarantee is a universal embedding theorem, showing that any $k$-point DAG can be embedded into an NST with $1+mathcal{O}(log(k))$ distortion while exactly preserving its causal structure. The total number of parameters defining the NST is sub-cubic in $k$ and linear in the width of the DAG. If the DAG has a planar Hasse diagram, this is improved to $mathcal{O}(log(k)) + 2)$ spatial and 2 temporal dimensions. We validate our framework computationally with synthetic weighted DAGs and real-world network embeddings; in both cases, the NSTs achieve lower embedding distortions than their counterparts using fixed spacetime geometries.

Problem

• Represent nodes in weighted DAGs as spacetime events.
• Encode graph edge weights and causality in spacetime dimensions.
• Achieve lower embedding distortions with data-driven spacetime geometries.

Innovation

• Neural Spacetimes combine space and time dimensions.
• Three neural networks optimize embeddings and geometries.
• Data-driven geometry shaping using fractal and deep learning.