Targeted synthetic data generation for tabular data via hardness characterization

📅 2024-10-01
🏛️ arXiv.org
📈 Citations: 0
Influential: 0
📄 PDF

career value

174K/year
🤖 AI Summary
To address the limited performance and poor robustness of models trained on few-shot or low-quality tabular data, this paper proposes a difficulty-aware targeted synthetic data generation method. Our core innovation is the first application of Shapley values to quantify sample-wise difficulty in tabular data, enabling precise identification and synthesis of high-value instances—thereby overcoming the redundancy inherent in conventional undirected augmentation techniques (e.g., SMOTE, CTGAN). The method integrates a hardness-driven synthetic strategy with lightweight learner fine-tuning. Extensive evaluation on benchmark datasets—including UCI and OpenML—demonstrates an average 3.2% improvement in downstream model AUC and a 47% reduction in training overhead. By jointly optimizing accuracy and efficiency, our approach significantly advances beyond the limitations of existing tabular data augmentation paradigms.

Technology Category

Application Category

📝 Abstract
Data augmentation via synthetic data generation has been shown to be effective in improving model performance and robustness in the context of scarce or low-quality data. Using the data valuation framework to statistically identify beneficial and detrimental observations, we introduce a simple augmentation pipeline that generates only high-value training points based on hardness characterization, in a computationally efficient manner. We first empirically demonstrate via benchmarks on real data that Shapley-based data valuation methods perform comparably with learning-based methods in hardness characterization tasks, while offering significant computational advantages. Then, we show that synthetic data generators trained on the hardest points outperform non-targeted data augmentation on a number of tabular datasets. Our approach improves the quality of out-of-sample predictions and it is computationally more efficient compared to non-targeted methods.
Problem

Research questions and friction points this paper is trying to address.

Generates high-value synthetic data efficiently
Improves model performance with scarce data
Enhances out-of-sample prediction quality
Innovation

Methods, ideas, or system contributions that make the work stand out.

Targeted synthetic data generation
Hardness characterization for augmentation
Shapley-based data valuation efficiency
🔎 Similar Papers
T
Tommaso Ferracci
Department of Mathematics, Imperial College London
L
L. Goldmann
Decision Science, Credit & Fraud Risk, American Express
A
Anton Hinel
Decision Science, Credit & Fraud Risk, American Express
F
Francesco Sanna Passino
Department of Mathematics, Imperial College London