Exploring the Heterogeneity of Tabular Data: A Diversity-aware Data Generator via LLMs

2025-12-26

AI Summary

Real-world heterogeneous tabular data generation faces challenges of poor model generalizability and difficulty balancing diversity with fidelity. Method: This paper proposes DATE—a framework comprising (1) distribution-aware data partitioning, (2) LLM–decision-tree co-guided controllable subset generation, and (3) multi-armed bandit (MAB)-based joint sampling to optimize both diversity and fidelity. Contribution/Results: Theoretically, we first prove that data selection in heterogeneous settings violates the greedy property. Methodologically, we introduce the first MAB-based joint sampling algorithm and establish a novel LLM–decision-tree collaborative reasoning paradigm. Experiments show that, using only 100 generated samples, DATE reduces average error rates by 23.75% on classification and regression benchmarks, significantly improves DPO training efficacy, and enhances LLM inference performance on target domains.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Tabular data generation has become increasingly essential for enabling robust machine learning applications, which require large-scale, high-quality data. Existing solutions leverage generative models to learn original data distributions. However, real-world data are naturally heterogeneous with diverse distributions, making it challenging to obtain a universally good model for diverse data generation. To address this limitation, we introduce Diversity-Aware Tabular data gEnerator (DATE), a framework that (i) prepares high-quality and distributionally distinct examples for in-context learning by effectively partitioning the original heterogeneous data into multiple diverse subsets; (ii) harnesses Large Language Models (LLMs) to explore the diversity of the partitioned distribution with decision tree reasoning as feedback, generating high-quality labeled data for each subset. However, the massive generated data inherently involves a trade-off between diversity and quality. To integrate this issue, existing solutions greedily select the validation-best data. However, we prove that the selection in heterogeneous settings does not possess the greedy-choice property, and design a Multi-Arm Bandit-based sampling algorithm that balances the diversity and quality of generated data. Extensive experiments on tabular classification and regression benchmarks demonstrate that DATE consistently outperforms state-of-the-art GAN-based and LLM-based methods. On average, DATE achieves a 23.75% reduction in error rate with just 100 generated data. Empirically, we demonstrate that data generated by DATE can improve the accuracy of Direct Preference Optimization (DPO) and enhance the reasoning capability of LLMs on the target data. Code is available at https://github.com/windblow32/DATE.

Problem

• Generates diverse, high-quality tabular data via LLMs
• Balances diversity and quality in heterogeneous data generation
• Improves machine learning model accuracy with synthetic data

Innovation

• Partitions heterogeneous data into diverse subsets for in-context learning
• Uses LLMs with decision tree feedback to generate labeled data per subset
• Employs Multi-Arm Bandit algorithm to balance diversity and quality