Systematic Relational Reasoning With Epistemic Graph Neural Networks

2024-07-24

AI Summary

Existing graph neural networks (GNNs) exhibit insufficient systematic generalization on long-chain reasoning tasks; conventional neuro-symbolic approaches, while logically grounded, suffer from restrictive single-path assumptions and poor scalability. This paper proposes Epistemic GNN (EpiGNN), the first GNN framework that models node embeddings as epistemic states and introduces a cognitive-logic-driven message-passing mechanism, enabling multi-path information aggregation and long-range dependency modeling. EpiGNN is parameter-efficient and scalable, eliminating reliance on strong single-relation-path assumptions. Experiments demonstrate that EpiGNN achieves state-of-the-art performance on systematic reasoning–oriented link prediction; matches specialized models in inductive knowledge graph completion; and significantly outperforms existing neuro-symbolic methods on a newly constructed multi-path aggregation benchmark.

Technology Category

• Knowledge Representation and Reasoning: Computational Complexity of Reasoning
• Machine Learning: Graph-based Machine Learning
• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Developing models that can learn to reason is a notoriously challenging problem. We focus on reasoning in relational domains, where the use of Graph Neural Networks (GNNs) seems like a natural choice. However, previous work has shown that regular GNNs lack the ability to systematically generalize from training examples on test graphs requiring longer inference chains, which fundamentally limits their reasoning abilities. A common solution relies on neuro-symbolic methods that systematically reason by learning rules, but their scalability is often limited and they tend to make unrealistically strong assumptions, e.g. that the answer can always be inferred from a single relational path. We propose the Epistemic GNN (EpiGNN), a novel parameter-efficient and scalable GNN architecture with an epistemic inductive bias for systematic reasoning. Node embeddings in EpiGNNs are treated as epistemic states, and message passing is implemented accordingly. We show that EpiGNNs achieve state-of-the-art results on link prediction tasks that require systematic reasoning. Furthermore, for inductive knowledge graph completion, EpiGNNs rival the performance of state-of-the-art specialized approaches. Finally, we introduce two new benchmarks that go beyond standard relational reasoning by requiring the aggregation of information from multiple paths. Here, existing neuro-symbolic approaches fail, yet EpiGNNs learn to reason accurately. Code and datasets are available at https://github.com/erg0dic/gnn-sg.

Problem

• Enhance systematic reasoning in relational domains using Graph Neural Networks.
• Address limitations of regular GNNs in generalizing from training examples.
• Propose scalable EpiGNN architecture for accurate multi-path relational reasoning.

Innovation

• Epistemic GNN for systematic relational reasoning
• Node embeddings as epistemic states in GNNs
• New benchmarks for multi-path information aggregation