Parameter-Free Encoders Remain Viable for RDB Foundation Models

2026-07-06

AI Summary

This work addresses the challenge of efficiently predicting missing or future values in target columns of relational databases without retraining a model from scratch for each new task. The authors propose a parameter-free subgraph encoder combined with a single-table base model, enabling effective multi-task learning when labels are provided as input. Theoretical analysis highlights fundamental limitations of trainable encoders under this setting. Remarkably, the proposed approach achieves performance close to state-of-the-art across multiple benchmark tasks without requiring database-specific pretraining, thereby demonstrating the efficacy and practicality of a simplified encoder design.

Abstract

Given a relational database (RDB) storing heterogeneous tabular information, how can we predict missing (or future) values in some target column of interest? As the space of potential targets is vast across enterprise settings, it is preferable to avoid learning a new model from scratch each time there is a new prediction task. Frozen foundation models based on RDB-specific encoders provide a viable solution, but ideal design remains an open question. On the one hand, it has recently been argued that certain parameter-free subgraph encoders combined with single-table foundation models can achieve near SOTA performance, with no RDB-specific pre-training required. Meanwhile, other contemporary studies advocate for parameterized encoders pre-trained to exploit observable labels for learning task-specific representations. To address this ambiguity, we analyze RDB encoder properties specifically when labels are present as inputs, proving limitations on the potential efficacy of trainable encoder parameters. As empirical validation, we demonstrate that considerably simpler parameter-free encoders are still capable of strong performance across many relevant benchmarking tasks.

Problem

• relational database
• foundation models
• parameter-free encoders
• missing value prediction
• encoder design

Innovation

• parameter-free encoders
• relational database foundation models
• frozen foundation models
• subgraph encoders
• tabular data prediction