MulTaBench: Benchmarking Multimodal Tabular Learning with Text and Image

2026-05-11

AI Summary

This work addresses the limitations of existing tabular foundation models, which natively lack support for unstructured modalities such as text and images, and the inadequacy of current benchmarks in evaluating task-aware representations. To bridge this gap, we introduce MulTaBench, the first large-scale multimodal tabular learning benchmark, comprising 40 datasets focused on image–table and text–table tasks where modalities are highly complementary yet generic embeddings often discard critical information. Spanning high-impact domains including healthcare and e-commerce, MulTaBench supports tunable modality-specific embeddings, diverse tabular learners, and encoders of varying scales, thereby enabling research into task-aware representations and joint multimodal modeling. Experiments demonstrate that fine-tuning task-aware representations consistently yields significant performance gains across modalities, architectures, and configurations, underscoring their generalizability and necessity.

Abstract

Tabular Foundation Models have recently established the state of the art in supervised tabular learning, by leveraging pretraining to learn generalizable representations of numerical and categorical structured data. However, they lack native support for unstructured modalities such as text and image, and rely on frozen, pretrained embeddings to process them. On established Multimodal Tabular Learning benchmarks, we show that tuning the embeddings to the task improves performance. Existing benchmarks, however, often focus on the mere co-occurrence of modalities; this leads to high variance across datasets and masks the benefits of task-specific tuning. To address this gap, we introduce MulTaBench, a benchmark of 40 datasets, split equally between image-tabular and text-tabular tasks. We focus on predictive tasks where the modalities provide complementary predictive signal, and where generic embeddings lose critical information, necessitating Target-Aware Representations that are aligned with the task. Our experimental results demonstrate that the gains from target-aware representation tuning generalize across both text and image modalities, several tabular learners, encoder scales, and embedding dimensions. MulTaBench constitutes the largest image-tabular benchmarking effort to date, spanning high-impact domains such as healthcare and e-commerce. It is designed to enable the research of novel architectures which incorporate joint modeling and target-aware representations, paving the way for the development of novel Multimodal Tabular Foundation Models.

Problem

• Multimodal Tabular Learning
• Tabular Foundation Models
• Target-Aware Representations
• Image-Tabular
• Text-Tabular

Innovation

• Multimodal Tabular Learning
• Target-Aware Representations
• MulTaBench
• Foundation Models
• Embedding Tuning