PathE: Leveraging Entity-Agnostic Paths for Parameter-Efficient Knowledge Graph Embeddings

2025-01-31

AI Summary

To address the parameter explosion and computational intractability arising from the rapidly increasing number of entities in large-scale knowledge graph embedding, this paper proposes an entity-agnostic path modeling paradigm: it eliminates explicit entity embedding tables and instead learns only lightweight relation embeddings, dynamically generating entity representations via multi-hop entity-relation paths. The core innovations include (i) a novel dynamic path aggregation encoding mechanism, (ii) a relation embedding sharing strategy across paths, and (iii) a lightweight path-contextualized neural network. Our method reduces model parameters by 75% compared to prior efficient baselines and enables end-to-end training on consumer-grade GPUs. It achieves state-of-the-art performance on relation prediction across four standard benchmarks and remains competitive on link prediction—particularly on path-rich knowledge graphs—while significantly improving both computational efficiency and scalability.

Technology Category

• Data Mining & Knowledge Management: Linked Open Data, Knowledge Graphs & KB Completion
• Reasoning under Uncertainty: Relational Probabilistic Models
• Knowledge Representation and Reasoning: Computational Complexity of Reasoning

Application Category

• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Knowledge Graphs (KGs) store human knowledge in the form of entities (nodes) and relations, and are used extensively in various applications. KG embeddings are an effective approach to addressing tasks like knowledge discovery, link prediction, and reasoning. This is often done by allocating and learning embedding tables for all or a subset of the entities. As this scales linearly with the number of entities, learning embedding models in real-world KGs with millions of nodes can be computationally intractable. To address this scalability problem, our model, PathE, only allocates embedding tables for relations (which are typically orders of magnitude fewer than the entities) and requires less than 25% of the parameters of previous parameter efficient methods. Rather than storing entity embeddings, we learn to compute them by leveraging multiple entity-relation paths to contextualise individual entities within triples. Evaluated on four benchmarks, PathE achieves state-of-the-art performance in relation prediction, and remains competitive in link prediction on path-rich KGs while training on consumer-grade hardware. We perform ablation experiments to test our design choices and analyse the sensitivity of the model to key hyper-parameters. PathE is efficient and cost-effective for relationally diverse and well-connected KGs commonly found in real-world applications.

Problem

• Knowledge Graph Embedding
• Efficiency
• Parameter Reduction

Innovation

• PathE model
• efficient knowledge graph embedding
• dynamic entity embedding