Universal Encoders for Modular Relational Deep Learning

2026-06-19

AI Summary

Existing relational deep learning approaches suffer from limited generalization, either requiring retraining for specific database schemas or employing monolithic architectures that tightly couple feature encoding with graph message passing. This work proposes a modular relational deep learning framework that decouples row encoding from graph message passing and introduces a universal row encoder. This encoder integrates cell values, column semantics, table names, and global statistics to produce table-width-invariant row embeddings, natively supporting sparse and previously unseen features. Built upon Transformers with intra-row self-attention and combined with heterogeneous temporal graph modeling, the proposed method achieves substantial improvements in cross-database generalization on the RelBench benchmark, while also accelerating convergence and reducing memory consumption.

Abstract

Relational Deep Learning (RDL) models multi-tabular databases as temporal heterogeneous graphs for end-to-end representation learning. While RDL is evolving rapidly, existing approaches face significant generalization obstacles. They are either schema-specific, requiring training from scratch for every new database, or they rely on monolithic architectures that entangle feature encoding with graph message-passing. Analyzing these limitations, we establish four core pillars for building foundational relational models: semantic granularity, structural topology, temporal causality, and unified optimization. Addressing these pillars, we propose a modular approach that decouples row encoding from graph message-passing. We introduce the Universal Row Encoder, a transformer-based module that integrates raw cell data with schema metadata$-$including column semantics, table names, and global distribution statistics$-$to produce table-width invariant row embeddings. By explicitly feeding global statistics to an intra-row self-attention mechanism, the encoder natively contextualizes unseen features and handles sparse data. Serving as a flexible "backend" for any downstream graph architecture, our pretrained encoder enhances cross-database knowledge transfer on the established RelBench benchmarks while improving learning convergence and memory footprint.

Problem

• Relational Deep Learning
• generalization
• schema-specific
• modular architecture
• cross-database transfer

Innovation

• Universal Row Encoder
• Modular Relational Deep Learning
• Schema Metadata Integration
• Table-width Invariant Embedding
• Cross-database Transfer