The Pitfalls of"Security by Obscurity"And What They Mean for Transparent AI

2025-01-30

AI Summary

This paper addresses the misconception in AI transparency initiatives that “security arises from obscurity,” drawing on cybersecurity’s long-standing rejection of “security through obscurity.” Method: Through interdisciplinary literature analysis, conceptual mapping modeling, and comparative case studies of anonymized implementations, it establishes the first cross-domain dialogue framework bridging security and AI transparency. Contribution/Results: The study proposes a novel three-dimensional trade-off model—encompassing explainability, auditability, and controllability—and identifies AI-specific barriers: data dependency, emergent behaviors, and black-box optimization. It delivers an actionable governance checklist and a practical guideline balancing open scrutiny with risk mitigation, thereby fostering collaborative R&D between security and AI communities on next-generation explainable and auditable AI technologies.

Technology Category

• Philosophy and Ethics of AI: Privacy & Security
• Computer Vision: Interpretability, Explainability, and Transparency
• Machine Learning: Transparent, Interpretable, Explainable ML

Application Category

• Security and Privacy: Data transparency and provenance
• Responsible Web: Algorithmic accountability and transparency on the web
• User Modeling, Personalization and Recommendation: Accountability, Transparency, and Ethics for personalization

Abstract

Calls for transparency in AI systems are growing in number and urgency from diverse stakeholders ranging from regulators to researchers to users (with a comparative absence of companies developing AI). Notions of transparency for AI abound, each addressing distinct interests and concerns. In computer security, transparency is likewise regarded as a key concept. The security community has for decades pushed back against so-called security by obscurity -- the idea that hiding how a system works protects it from attack -- against significant pressure from industry and other stakeholders. Over the decades, in a community process that is imperfect and ongoing, security researchers and practitioners have gradually built up some norms and practices around how to balance transparency interests with possible negative side effects. This paper asks: What insights can the AI community take from the security community's experience with transparency? We identify three key themes in the security community's perspective on the benefits of transparency and their approach to balancing transparency against countervailing interests. For each, we investigate parallels and insights relevant to transparency in AI. We then provide a case study discussion on how transparency has shaped the research subfield of anonymization. Finally, shifting our focus from similarities to differences, we highlight key transparency issues where modern AI systems present challenges different from other kinds of security-critical systems, raising interesting open questions for the security and AI communities alike.

Problem

• AI Transparency
• Security Risks
• Computer Security

Innovation

• AI Transparency
• Security by Obscurity
• Anonymization Technologies