Representation Learning for Tabular Data: A Comprehensive Survey

2025-04-17

AI Summary

This survey addresses fundamental challenges in tabular data representation learning—namely, heterogeneous feature coupling, sample sparsity, and weak task generalization—hindering effective modeling of structured tabular information with deep neural networks. To tackle these issues, we propose a three-tiered methodological framework—*specialized*, *transferable*, and *general-purpose*—and introduce the first taxonomy categorizing models along three dimensions: features, samples, and prediction objectives. We formally define and classify transferable models and tabular foundation models, and unify recent advances in multimodal alignment, open-world adaptation, and self-supervised pretraining. Our work yields the first structured, extensible landscape of tabular representation learning, accompanied by an open-source repository (GitHub) containing curated resources, benchmarks, and implementation guidelines. This synthesis provides both theoretical foundations and practical blueprints for algorithm design and industrial deployment. (138 words)

Technology Category

• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Knowledge Representation and Reasoning: Ontologies
• Reasoning under Uncertainty: Relational Probabilistic Models

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods, algorithms and applications for the development of semantic models, knowledge graphs and other forms of structured data models with machine-interpretable semantics
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Tabular data, structured as rows and columns, is among the most prevalent data types in machine learning classification and regression applications. Models for learning from tabular data have continuously evolved, with Deep Neural Networks (DNNs) recently demonstrating promising results through their capability of representation learning. In this survey, we systematically introduce the field of tabular representation learning, covering the background, challenges, and benchmarks, along with the pros and cons of using DNNs. We organize existing methods into three main categories according to their generalization capabilities: specialized, transferable, and general models. Specialized models focus on tasks where training and evaluation occur within the same data distribution. We introduce a hierarchical taxonomy for specialized models based on the key aspects of tabular data -- features, samples, and objectives -- and delve into detailed strategies for obtaining high-quality feature- and sample-level representations. Transferable models are pre-trained on one or more datasets and subsequently fine-tuned on downstream tasks, leveraging knowledge acquired from homogeneous or heterogeneous sources, or even cross-modalities such as vision and language. General models, also known as tabular foundation models, extend this concept further, allowing direct application to downstream tasks without fine-tuning. We group these general models based on the strategies used to adapt across heterogeneous datasets. Additionally, we explore ensemble methods, which integrate the strengths of multiple tabular models. Finally, we discuss representative extensions of tabular learning, including open-environment tabular machine learning, multimodal learning with tabular data, and tabular understanding. More information can be found in the following repository: https://github.com/LAMDA-Tabular/Tabular-Survey.

Problem

• Surveying representation learning methods for tabular data
• Comparing DNN-based models for tabular data classification
• Organizing models by generalization: specialized, transferable, general

Innovation

• Deep Neural Networks for tabular representation learning
• Hierarchical taxonomy for specialized tabular models
• General tabular models without fine-tuning