Large Language Models are Good Relational Learners

2025-06-06

AI Summary

Existing relational deep learning (RDL) approaches flatten structured data into textual sequences for large language models (LLMs), leading to loss of relational topology, semantic redundancy, and context-length overflow. To address this, we propose Rel-LLM—a RAG-based architecture integrating a graph neural network (GNN) encoder that directly generates structured subgraph prompts from relational data, explicitly encoding both topological and temporal dependencies among entities. Its core innovation is a GNN-driven structured prompting mechanism, which preserves relational structure—bypassing the structural distortion inherent in sequentialization—and enables end-to-end joint reasoning over subgraph-level relational and temporal patterns by the LLM. Extensive experiments across diverse RDL tasks demonstrate significant improvements over state-of-the-art methods, achieving superior accuracy, context efficiency, and database scalability. The implementation is publicly available.

Technology Category

• Machine Learning: Statistical Relational/Logic Learning
• Data Mining & Knowledge Management: Linked Open Data, Knowledge Graphs & KB Completion
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Large language models (LLMs) have demonstrated remarkable capabilities across various domains, yet their application to relational deep learning (RDL) remains underexplored. Existing approaches adapt LLMs by traversing relational links between entities in a database and converting the structured data into flat text documents. Still, this text-based serialization disregards critical relational structures, introduces redundancy, and often exceeds standard LLM context lengths. We introduce Rel-LLM, a novel architecture that utilizes a graph neural network (GNN)- based encoder to generate structured relational prompts for LLMs within a retrieval-augmented generation (RAG) framework. Unlike traditional text-based serialization approaches, our method preserves the inherent relational structure of databases while enabling LLMs to effectively process and reason over complex entity relationships. Specifically, the GNN encoder extracts a local subgraph around an entity to build feature representations that contain relevant entity relationships and temporal dependencies. These representations are transformed into structured prompts using a denormalization process, effectively allowing the LLM to reason over relational structures. Through extensive experiments, we demonstrate that Rel-LLM outperforms existing methods on key RDL tasks, offering a scalable and efficient approach to integrating LLMs with structured data sources. Code is available at https://github.com/smiles724/Rel-LLM.

Problem

• LLMs struggle with relational deep learning due to text serialization limitations
• Existing methods lose critical relational structures in databases
• Need scalable integration of LLMs with structured data sources

Innovation

• GNN encoder generates structured relational prompts
• Retrieval-augmented framework preserves database relationships
• Denormalization transforms subgraphs into LLM-readable prompts