Defending Against Unforeseen Failure Modes with Latent Adversarial Training

2024-03-08

arXiv.org

AI Summary

AI systems often exhibit harmful behaviors due to unknown failure modes, yet conventional red-teaming and adversarial training rely on known attack samples, limiting generalization to unseen threats. Method: We propose Latent-space Adversarial Training (LAT), the first approach to inject adversarial perturbations directly into structured latent representations—without requiring prior knowledge of attack patterns or concrete adversarial examples. LAT integrates latent-space modeling, adversarial perturbation of latent variables, and multi-task joint optimization, and is applicable to image classification, text classification, and text generation. Contribution/Results: Experiments demonstrate that LAT significantly improves robustness against unseen attack types, backdoor triggers, and out-of-distribution adversarial examples, while preserving or even enhancing clean-sample accuracy. By decoupling adversarial defense from reliance on known attack templates, LAT establishes a general-purpose defense framework for previously unobserved failure modes.

Technology Category

• Machine Learning: Adversarial Learning & Robustness
• Computer Vision: Adversarial Attacks & Robustness
• Multiagent Systems: Adversarial Agents

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Despite extensive diagnostics and debugging by developers, AI systems sometimes exhibit harmful unintended behaviors. Finding and fixing these is challenging because the attack surface is so large -- it is not tractable to exhaustively search for inputs that may elicit harmful behaviors. Red-teaming and adversarial training (AT) are commonly used to improve robustness, however, they empirically struggle to fix failure modes that differ from the attacks used during training. In this work, we utilize latent adversarial training (LAT) to defend against vulnerabilities without leveraging knowledge of what they are or using inputs that elicit them. LAT makes use of the compressed, abstract, and structured latent representations of concepts that the network actually uses for prediction. Here, we use it to defend against failure modes without examples that elicit them. Specifically, we use LAT to remove trojans and defend against held-out classes of adversarial attacks. We show in image classification, text classification, and text generation tasks that LAT usually improves both robustness to novel attacks and performance on clean data relative to AT. This suggests that LAT can be a promising tool for defending against failure modes that are not explicitly identified by developers.

Problem

• Defending AI against unforeseen harmful behaviors
• Improving robustness without known attack examples
• Removing backdoors and resisting novel adversarial attacks

Innovation

• Uses latent adversarial training (LAT)
• Defends without known vulnerabilities examples
• Improves robustness and clean data performance