Towards Evaluating Data Priors for Tabular Foundation Models

2026-06-28

AI Summary

Existing tabular foundation models lack an independent evaluation of the impact of data priors, making it difficult to disentangle their effects from those of model architecture or training protocols. This work proposes a unified interface that decouples data priors from architectural and training choices, enabling systematic generation of tasks under fixed conditions and subsequent evaluation of downstream classification performance. For the first time, data priors are treated as an isolated variable in a controlled comparative study, revealing their distinct influence on model behavior: different priors yield significantly divergent downstream performance, with some achieving higher absolute accuracy while others exhibit more stable relative rankings across multiple datasets. Notably, data similarity alone only partially accounts for these observed performance differences.

Abstract

Data-generating priors are a central component of tabular foundation models because they define the task distribution used during pretraining. However, priors are rarely evaluated as independent components, making it difficult to understand how much they affect downstream model behavior. This raises a methodological question: how can priors from different tabular foundation models be compared independently of the architectures and training protocols they were introduced with? To study this question, we implement a unified interface for publicly available priors from recent tabular foundation models and priors constructed from real datasets. We generate training tasks from each prior, train the same model architecture under a fixed training protocol, and evaluate the resulting models on shared downstream classification tasks. We compare priors through both generated-task statistics and downstream predictive performance. Our results show that different priors favor different downstream behaviors, with some achieving stronger absolute performance and others exhibiting more consistent relative rankings across datasets. We further find that data-level similarity only partially explains downstream behavior. Our code is available at https://github.com/automl/TFM-Playground/tree/prior-dev.

Problem

• tabular foundation models
• data priors
• prior evaluation
• downstream performance
• task distribution

Innovation

• data priors
• tabular foundation models
• prior evaluation
• unified benchmarking
• downstream performance