Two-Stage Data Synthesization: A Statistics-Driven Restricted Trade-off between Privacy and Prediction

2026-02-09

AI Summary

Existing data synthesis methods struggle to balance strong privacy perturbations with the data fidelity required for downstream tasks. This work proposes a novel two-stage synthesis mechanism: in the first stage, a post-synthesis mixing strategy blends original and synthetic data, while the second stage employs a kernel ridge regression model to generate outputs, jointly preserving covariate distributions and optimizing predictive performance. The approach achieves, for the first time, a theoretically grounded, statistically driven trade-off between privacy and utility, overcoming limitations inherent in single-stage methods. Experimental results demonstrate its superior generalization capability and prediction accuracy across five real-world datasets and a marketing scenario.

Technology Category

• Machine Learning: Privacy
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Search and Optimization: Mixed Discrete/Continuous Search

Application Category

• User Modeling, Personalization and Recommendation: User privacy protection in personalized systems
• Security and Privacy: Data transparency and provenance
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI

Abstract

Synthetic data have gained increasing attention across various domains, with a growing emphasis on their performance in downstream prediction tasks. However, most existing synthesis strategies focus on maintaining statistical information. Although some studies address prediction performance guarantees, their single-stage synthesis designs make it challenging to balance the privacy requirements that necessitate significant perturbations and the prediction performance that is sensitive to such perturbations. We propose a two-stage synthesis strategy. In the first stage, we introduce a synthesis-then-hybrid strategy, which involves a synthesis operation to generate pure synthetic data, followed by a hybrid operation that fuses the synthetic data with the original data. In the second stage, we present a kernel ridge regression (KRR)-based synthesis strategy, where a KRR model is first trained on the original data and then used to generate synthetic outputs based on the synthetic inputs produced in the first stage. By leveraging the theoretical strengths of KRR and the covariant distribution retention achieved in the first stage, our proposed two-stage synthesis strategy enables a statistics-driven restricted privacy--prediction trade-off and guarantee optimal prediction performance. We validate our approach and demonstrate its characteristics of being statistics-driven and restricted in achieving the privacy--prediction trade-off both theoretically and numerically. Additionally, we showcase its generalizability through applications to a marketing problem and five real-world datasets.

Problem

• synthetic data
• privacy
• prediction performance
• data synthesis
• privacy-prediction trade-off

Innovation

• two-stage synthesis
• privacy–prediction trade-off
• kernel ridge regression
• synthetic data
• statistics-driven