Hidden Threat in Synthetic Data: Covert Targeted Bias Injection through Benign Text

2026-08-31

AI Summary

研究探讨通过看似无害的合成数据向语言模型注入定向社会偏见的问题，构建了一种利用不一致教师模型生成的数据来微调学生模型的方法，并提出基于对数线性的评分作为潜在的筛查手段。

Abstract

Synthetic data is increasingly used to train large language models (LLMs), yet its security implications remain poorly understood. Prior work on subliminal learning suggests that models can inherit behavioral traits from seemingly unrelated training data. In this work, we investigate whether such mechanisms can be exploited to inject targeted social biases into aligned models through semantically benign synthetic data. We construct a pipeline in which a misaligned teacher model generates filtered synthetic datasets across domains such as creative writing and code generation, which are then used to fine-tune aligned student models. Our experiments show that benign-looking synthetic data can act as a covert channel for transmitting targeted biases while largely preserving the student model's general task capabilities. These results reveal a previously underexplored security risk in synthetic data-driven LLM training pipelines and highlight the need for improved safeguards. As one possible step toward this goal, we suggest that log-linearity-based scoring may provide a useful signal for screening seemingly benign synthetic data.

Problem

• synthetic data
• targeted bias
• large language models
• security risk

Innovation

• synthetic data
• targeted bias injection
• large language models
• covert channel
• log-linearity-based scoring