Continual Contrastive Learning on Tabular Data with Out of Distribution

2025-03-19

AI Summary

To address the poor out-of-distribution (OOD) generalization of tabular models under distributional shift, this paper proposes Tabular Continual Contrastive Learning (TCCL)—the first framework to synergistically integrate contrastive learning with continual learning. TCCL introduces an incrementally updatable encoder–decoder–learning-head architecture that enables robust representation evolution in response to dynamic distribution drift. The method jointly optimizes self-supervised autoencoding reconstruction, representation alignment, and multi-task learning, supporting online OOD detection and regression. Evaluated on eight real-world tabular datasets, TCCL consistently outperforms 14 state-of-the-art baselines—including gradient-boosted decision trees and leading deep models—achieving superior performance in both OOD classification and regression. Moreover, it demonstrates significantly enhanced generalization capability and training stability under continual distribution shifts.

Technology Category

• Machine Learning: Life-Long and Continual Learning
• Search and Optimization: Learning to Search
• Constraint Satisfaction and Optimization: Constraint Learning and Acquisition

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Out-of-distribution (OOD) prediction remains a significant challenge in machine learning, particularly for tabular data where traditional methods often fail to generalize beyond their training distribution. This paper introduces Tabular Continual Contrastive Learning (TCCL), a novel framework designed to address OOD challenges in tabular data processing. TCCL integrates contrastive learning principles with continual learning mechanisms, featuring a three-component architecture: an Encoder for data transformation, a Decoder for representation learning, and a Learner Head. We evaluate TCCL against 14 baseline models, including state-of-the-art deep learning approaches and gradient-boosted decision trees (GBDT), across eight diverse tabular datasets. Our experimental results demonstrate that TCCL consistently outperforms existing methods in both classification and regression tasks on OOD data, with particular strength in handling distribution shifts. These findings suggest that TCCL represents a significant advancement in handling OOD scenarios for tabular data.

Problem

• Addresses out-of-distribution prediction challenges in tabular data.
• Introduces a novel framework combining contrastive and continual learning.
• Outperforms existing methods in classification and regression on OOD data.

Innovation

• Integrates contrastive and continual learning
• Three-component architecture: Encoder, Decoder, Learner
• Outperforms 14 baselines on OOD tabular data