Training Fair Tabular Foundation Models

2026-08-14

AI Summary

This study addresses the lack of fairness guarantees in tabular foundation models for high-stakes decision-making and the incompatibility of existing methods with in-context learning when sensitive attributes are unavailable. We propose FairTFM, a novel fair training strategy that embeds fairness constraints directly into the training process via synthetic fair tasks and gradient reversal layers. This approach enables representation debiasing and fair prediction within a single forward pass. Extensive experiments across 132 tasks demonstrate that FairTFM significantly improves fairness metrics while maintaining competitive predictive accuracy. Consequently, this work effectively resolves the challenges associated with restricted access to sensitive attributes and incompatibility with in-context learning, providing a robust solution for deploying equitable tabular foundation models in critical applications.

Abstract

Tabular Foundation Models (TFMs) have emerged as leading methods for tabular predictive tasks, leveraging in-context learning to predict on new data without task-specific training. Despite the increased use of TFMs in high-stakes decision-making, their fairness properties remain largely unexplored. In this work, we incorporate fairness constraints directly into TFM training, enabling fair predictions in a single forward pass. Our approach addresses two key challenges: limited access to sensitive attributes in training data, and the incompatibility of existing fairness techniques with the in-context learning paradigm. We propose FairTFM, a scalable training strategy based on synthetic fairness tasks and a fairness-aware architecture using a gradient reversal layer, which encourages the model to learn representations invariant to sensitive attributes. Experiments on 132 fairness tasks show consistent improvements in fairness while maintaining competitive accuracy.

Problem

• Tabular Foundation Models
• Fairness
• In-context Learning
• Sensitive Attributes

Innovation

• Tabular Foundation Models
• Fairness Constraints
• Gradient Reversal Layer
• Synthetic Fairness Tasks
• Invariant Representations