Tabular Data Adapters: Improving Outlier Detection for Unlabeled Private Data

2025-04-29

AI Summary

To address the cold-start challenge in anomaly detection on private, unlabeled tabular data—caused by domain shift, structural heterogeneity, and absence of annotations—this paper proposes Tabular Data Adapters (TDA). TDA first retrieves statistically similar public datasets, then aligns cross-domain representations via a shared autoencoder, and finally generates high-quality soft labels through weak supervision—enabling reuse of public state-of-the-art models without manual annotation of private data. TDA is the first unified framework supporting both cross-domain representation alignment and weakly supervised label generation. Extensive experiments across 50 cross-domain datasets demonstrate that TDA achieves significantly higher soft-label accuracy than baselines, reduces inference latency, and offers industrial-scale scalability and deployment cost efficiency.

Technology Category

• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Data Mining & Knowledge Management: Anomaly/Outlier Detection
• Application Domains: Humanities & Computational Social Science

Application Category

• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Responsible Web: Data and user privacy-enhancing technologies for the Web
• Security and Privacy: Data transparency and provenance

Abstract

The remarkable success of Deep Learning approaches is often based and demonstrated on large public datasets. However, when applying such approaches to internal, private datasets, one frequently faces challenges arising from structural differences in the datasets, domain shift, and the lack of labels. In this work, we introduce Tabular Data Adapters (TDA), a novel method for generating soft labels for unlabeled tabular data in outlier detection tasks. By identifying statistically similar public datasets and transforming private data (based on a shared autoencoder) into a format compatible with state-of-the-art public models, our approach enables the generation of weak labels. It thereby can help to mitigate the cold start problem of labeling by basing on existing outlier detection models for public datasets. In experiments on 50 tabular datasets across different domains, we demonstrate that our method is able to provide more accurate annotations than baseline approaches while reducing computational time. Our approach offers a scalable, efficient, and cost-effective solution, to bridge the gap between public research models and real-world industrial applications.

Problem

• Detecting outliers in unlabeled private tabular data
• Addressing domain shift and lack of labels in private datasets
• Bridging public models and industrial applications efficiently

Innovation

• Generates soft labels for unlabeled tabular data
• Transforms private data via shared autoencoder
• Leverages public datasets for outlier detection