AdapTable: Test-Time Adaptation for Tabular Data via Shift-Aware Uncertainty Calibrator and Label Distribution Handler

2024-07-15

arXiv.org

AI Summary

Real-world tabular data often suffer from distribution shifts—particularly label distribution shift (LDS)—leading to substantial degradation in model performance; existing test-time adaptation (TTA) methods either neglect LDS characteristics or are constrained by architectural assumptions, limiting applicability in privacy-sensitive settings. This paper proposes the first source-free TTA framework for tabular data, introducing a novel two-stage decoupled mechanism: an offset-aware uncertainty calibrator jointly optimizes with a label distribution processor to systematically model and mitigate LDS for the first time. The method integrates uncertainty calibration, label distribution estimation and reweighting, theory-guided robustness analysis, and source-free online adaptation. Evaluated on diverse LDS benchmarks—including HELOC—it achieves up to 16% accuracy improvement over state-of-the-art TTA approaches, demonstrating strong model-agnosticism and generalization capability.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Search and Optimization: Distributed Search
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems

Abstract

In real-world scenarios, tabular data often suffer from distribution shifts that threaten the performance of machine learning models. Despite its prevalence and importance, handling distribution shifts in the tabular domain remains underexplored due to the inherent challenges within the tabular data itself. In this sense, test-time adaptation (TTA) offers a promising solution by adapting models to target data without accessing source data, crucial for privacy-sensitive tabular domains. However, existing TTA methods either 1) overlook the nature of tabular distribution shifts, often involving label distribution shifts, or 2) impose architectural constraints on the model, leading to a lack of applicability. To this end, we propose AdapTable, a novel TTA framework for tabular data. AdapTable operates in two stages: 1) calibrating model predictions using a shift-aware uncertainty calibrator, and 2) adjusting these predictions to match the target label distribution with a label distribution handler. We validate the effectiveness of AdapTable through theoretical analysis and extensive experiments on various distribution shift scenarios. Our results demonstrate AdapTable's ability to handle various real-world distribution shifts, achieving up to a 16% improvement on the HELOC dataset.

Problem

• Handling distribution shifts in tabular data
• Test-time adaptation without source data access
• Calibrating predictions for label distribution shifts

Innovation

• Test-time adaptation for tabular data
• Shift-aware uncertainty calibrator
• Label distribution handler