Leveraging Graph Structure in Seq2Seq Models for Knowledge Graph Link Prediction

2026-05-18

AI Summary

This work addresses the limitation of existing sequence-to-sequence (Seq2Seq) models in knowledge graph link prediction, which often ignore graph structure and struggle to capture multi-hop relational patterns. To overcome this, the authors propose a Graph-Augmented Sequence-to-Sequence (GA-S2S) framework that uniquely integrates a Relational Graph Attention Network (RGAT) with a T5-small encoder–decoder architecture. This integration enables joint encoding of entity textual features and k-hop subgraph topological structures, preserving local graph integrity while transcending the constraints of conventional sequential modeling. Experimental results demonstrate that the proposed approach achieves up to a 19% improvement in link prediction accuracy over strong baselines on the CoDEx dataset.

Abstract

We introduce Graph-Augmented Sequence-to-Sequence (GA-S2S), a novel framework that integrates a T5-small encoder-decoder with a Relational Graph Attention Network (RGAT) to improve link prediction in knowledge graphs. While existing Seq2Seq models rely solely on surface-level textual descriptions of entities and relations and at best, flatten the neighborhoods of a query entity into a single linear sequence, thereby discarding the inherent graph structure, GA-S2S jointly encodes both textual features and the full $k$-hop subgraph topology surrounding the query entity. By integrating raw encoder outputs with RGAT's relation-aware embeddings, our model captures and leverages richer multi-hop relational patterns and textual information. Our preliminary experiments on the CoDEx dataset demonstrate that GA-S2S outperforms competitive Seq2Seq-based baseline models, achieving up to a 19\% relative gain in link prediction accuracy.

Problem

• Knowledge Graph Link Prediction
• Seq2Seq Models
• Graph Structure
• Relational Patterns
• Entity Neighborhood

Innovation

• Graph-Augmented Sequence-to-Sequence
• Relational Graph Attention Network
• knowledge graph link prediction
• multi-hop relational patterns
• graph structure integration