Boosting Relational Deep Learning with Pretrained Tabular Models

2025-04-07

AI Summary

Graph neural networks (GNNs) suffer from high inference latency in relational database prediction, while hand-crafted features struggle to capture complex structural dependencies. Method: We propose LightRDL, a lightweight framework that synergistically integrates pretrained tabular models with GNNs for relational learning—decoupling responsibilities such that the GNN exclusively models structural dependencies among entities, while the tabular model efficiently encodes temporal and attribute features, eliminating the need for constructing and storing full historical graphs. Contribution/Results: LightRDL significantly reduces inference overhead while preserving expressive relational pattern modeling. On the RelBench benchmark, it achieves a 33% improvement in prediction accuracy over baseline GNNs and accelerates inference by 526×, thereby greatly enhancing feasibility for real-time prediction scenarios.

Technology Category

• Machine Learning: Statistical Relational/Logic Learning
• Reasoning under Uncertainty: Relational Probabilistic Models
• Knowledge Representation and Reasoning: Description Logics

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Relational databases, organized into tables connected by primary-foreign key relationships, are a common format for organizing data. Making predictions on relational data often involves transforming them into a flat tabular format through table joins and feature engineering, which serve as input to tabular methods. However, designing features that fully capture complex relational patterns remains challenging. Graph Neural Networks (GNNs) offer a compelling alternative by inherently modeling these relationships, but their time overhead during inference limits their applicability for real-time scenarios. In this work, we aim to bridge this gap by leveraging existing feature engineering efforts to enhance the efficiency of GNNs in relational databases. Specifically, we use GNNs to capture complex relationships within relational databases, patterns that are difficult to featurize, while employing engineered features to encode temporal information, thereby avoiding the need to retain the entire historical graph and enabling the use of smaller, more efficient graphs. Our extsc{LightRDL} approach not only improves efficiency, but also outperforms existing models. Experimental results on the RelBench benchmark demonstrate that our framework achieves up to $33%$ performance improvement and a $526 imes$ inference speedup compared to GNNs, making it highly suitable for real-time inference.

Problem

• Enhancing GNN efficiency in relational databases
• Capturing complex relational patterns without full feature engineering
• Enabling real-time inference with improved speed and performance

Innovation

• Combines GNNs with engineered features
• Uses small graphs for efficiency
• Achieves fast real-time inference