Cascading Adversarial Bias from Injection to Distillation in Language Models

2025-05-30

AI Summary

This work identifies a cascading amplification phenomenon of adversarial bias in knowledge distillation: minimal data poisoning (just 0.25%) in the teacher model triggers a dramatic surge in student model bias response rate—to 76.9%, up to 6–29× higher than the teacher’s. We first categorize bias propagation into *undirected* and *directed* modes, and propose a novel cross-modal bias injection framework covering advertising manipulation, phishing, narrative control, and unsafe code generation. Systematic evaluation across six bias types, multiple distillation paradigms (e.g., response-level, logits-level), and text/code generation tasks reveals that mainstream defenses—including perplexity filtering, bias detection classifiers, and LLM self-evaluation—fail entirely. Our findings provide critical empirical evidence and a new analytical framework for understanding bias propagation mechanisms in large language models and enhancing the robustness of knowledge distillation pipelines.

Technology Category

• Machine Learning: Adversarial Learning & Robustness
• Computer Vision: Adversarial Attacks & Robustness
• Natural Language Processing: Ethics — Bias, Fairness, Transparency & Privacy

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Model distillation has become essential for creating smaller, deployable language models that retain larger system capabilities. However, widespread deployment raises concerns about resilience to adversarial manipulation. This paper investigates vulnerability of distilled models to adversarial injection of biased content during training. We demonstrate that adversaries can inject subtle biases into teacher models through minimal data poisoning, which propagates to student models and becomes significantly amplified. We propose two propagation modes: Untargeted Propagation, where bias affects multiple tasks, and Targeted Propagation, focusing on specific tasks while maintaining normal behavior elsewhere. With only 25 poisoned samples (0.25% poisoning rate), student models generate biased responses 76.9% of the time in targeted scenarios - higher than 69.4% in teacher models. For untargeted propagation, adversarial bias appears 6x-29x more frequently in student models on unseen tasks. We validate findings across six bias types (targeted advertisements, phishing links, narrative manipulations, insecure coding practices), various distillation methods, and different modalities spanning text and code generation. Our evaluation reveals shortcomings in current defenses - perplexity filtering, bias detection systems, and LLM-based autorater frameworks - against these attacks. Results expose significant security vulnerabilities in distilled models, highlighting need for specialized safeguards. We propose practical design principles for building effective adversarial bias mitigation strategies.

Problem

• Investigates vulnerability of distilled models to adversarial bias injection
• Demonstrates bias amplification from teacher to student models
• Exposes shortcomings in current defenses against adversarial attacks

Innovation

• Minimal data poisoning injects subtle biases
• Bias amplifies during model distillation process
• Proposes targeted and untargeted bias propagation modes