P-TA: Using Proximal Policy Optimization to Enhance Tabular Data Augmentation via Large Language Models

2024-06-17

Annual Meeting of the Association for Computational Linguistics

AI Summary

To address the limitations of GANs—lacking commonsense knowledge—and LLMs—struggling to align with real data distributions and suffering from gradient truncation during discrete token decoding—this paper proposes PPO-GAN, the first framework to integrate Proximal Policy Optimization (PPO) into tabular data augmentation. It embeds an LLM as a differentiable policy network within a GAN architecture, enabling reinforcement learning–driven parameter updates guided by discriminator feedback. By treating LLM generation as a continuous policy optimization problem, PPO-GAN overcomes the non-differentiability barrier inherent in autoregressive token sampling, thereby unifying knowledge-guided synthesis with distributional fidelity. Evaluated on three real-world tabular datasets, downstream models trained on PPO-GAN–generated synthetic data achieve ~4% higher accuracy than state-of-the-art methods, with significant improvements in semantic coherence and statistical quality.

Technology Category

• Natural Language Processing: Generation
• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Generative Adversarial Networks (GANs) for Vision

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

A multitude of industries depend on accurate and reasonable tabular data augmentation for their business processes. Contemporary methodologies in generating tabular data revolve around utilizing Generative Adversarial Networks (GAN) or fine-tuning Large Language Models (LLM). However, GAN-based approaches are documented to produce samples with common-sense errors attributed to the absence of external knowledge. On the other hand, LLM-based methods exhibit a limited capacity to capture the disparities between synthesized and actual data distribution due to the absence of feedback from a discriminator during training. Furthermore, the decoding of LLM-based generation introduces gradient breakpoints, impeding the backpropagation of loss from a discriminator, thereby complicating the integration of these two approaches. To solve this challenge, we propose using proximal policy optimization (PPO) to apply GANs, guiding LLMs to enhance the probability distribution of tabular features. This approach enables the utilization of LLMs as generators for GANs in synthesizing tabular data. Our experiments demonstrate that PPO leads to an approximately 4% improvement in the accuracy of models trained on synthetically generated data over state-of-the-art across three real-world datasets.

Problem

• Improving tabular data augmentation accuracy
• Integrating GANs and LLMs effectively
• Reducing common-sense errors in synthesized data

Innovation

• Proximal Policy Optimization (PPO) integration
• Enhanced tabular data augmentation
• Guided Large Language Models (LLMs)