EPSVec: Efficient and Private Synthetic Data Generation via Dataset Vectors
This work addresses the inefficiency and high computational cost of existing privacy-preserving text generation methods in low-data regimes, which often fail to produce high-quality synthetic data. The authors propose a novel approach that constructs a differentially private “dataset vector” to capture the distributional discrepancy between private data and public priors in the activation space, leveraging this vector to guide large language models during text generation. By integrating dataset vectors with differential privacy for the first time, the method decouples the privacy budget from the generation process, enabling unlimited text synthesis without incurring additional privacy loss. Experiments demonstrate that the proposed method significantly outperforms current baselines under low-data conditions, achieving high fidelity in both distribution alignment and downstream task utility while substantially reducing computational overhead.