On the Trade-Off Between Transparency and Security in Adversarial Machine Learning

2025-11-14

AI Summary

This work investigates the fundamental tension between model transparency and system security in adversarial machine learning. Focusing on transferable adversarial example attacks, we formulate a game-theoretic framework—incorporating both Nash and Stackelberg equilibria—between attackers and defenders, and conduct large-scale empirical evaluation across nine attack methods and 181 models. Our analysis reveals that transparency substantially increases attack success: merely knowing whether a model is protected suffices to significantly degrade its robustness. We introduce “stealthiness” as a critical defensive advantage, formally demonstrating that strategically concealing defense mechanisms yields superior security outcomes compared to full transparency. To our knowledge, this is the first study to formalize the transparency–security trade-off using rigorous game-theoretic analysis. The findings provide both theoretical grounding and practical guidance for deploying responsible AI systems under adversarial conditions.

Technology Category

• Machine Learning: Adversarial Learning & Robustness
• Computer Vision: Adversarial Attacks & Robustness
• Game Theory and Economic Paradigms: Adversarial Learning

Application Category

• Security and Privacy: Data transparency and provenance
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Responsible Web: Algorithmic accountability and transparency on the web

Abstract

Transparency and security are both central to Responsible AI, but they may conflict in adversarial settings. We investigate the strategic effect of transparency for agents through the lens of transferable adversarial example attacks. In transferable adversarial example attacks, attackers maliciously perturb their inputs using surrogate models to fool a defender's target model. These models can be defended or undefended, with both players having to decide which to use. Using a large-scale empirical evaluation of nine attacks across 181 models, we find that attackers are more successful when they match the defender's decision; hence, obscurity could be beneficial to the defender. With game theory, we analyze this trade-off between transparency and security by modeling this problem as both a Nash game and a Stackelberg game, and comparing the expected outcomes. Our analysis confirms that only knowing whether a defender's model is defended or not can sometimes be enough to damage its security. This result serves as an indicator of the general trade-off between transparency and security, suggesting that transparency in AI systems can be at odds with security. Beyond adversarial machine learning, our work illustrates how game-theoretic reasoning can uncover conflicts between transparency and security.

Problem

• Investigating transparency-security trade-off in adversarial machine learning systems
• Analyzing transferable adversarial attacks using game theory models
• Demonstrating how model transparency can compromise security defenses

Innovation

• Uses transferable adversarial example attacks analysis
• Models transparency-security trade-off with game theory
• Empirically evaluates attacks across 181 different models