🤖 AI Summary
This work addresses the critical risk of loss of control (LoC) in artificial intelligence systems deployed in high-stakes domains such as national security, where unintended behaviors could lead to severe consequences. The authors propose an empirical safety methodology grounded in task-specific benchmarks: by analyzing erroneous AI responses on national security–relevant benchmark tasks, they trace back to the high-risk permissions and functionalities underlying these failures and apply selective interventions. This approach disrupts potential harm pathways while preserving the system’s capacity for correct behavior. Innovatively linking benchmark errors directly to LoC mitigation mechanisms, the method enables deployers to enact proactive, evidence-based controls derived from real-world usage. Experimental validation on a derived classified document categorization task demonstrates the strategy’s efficacy, offering an immediately deployable solution for mitigating LoC in high-risk AI systems.
📝 Abstract
Affordances and permissions are promising and timely safety levers for mitigating Loss of Control (LoC) threats in high-stakes deployment contexts, such as national security. Deployers in defense and intelligence could rely on several approaches to identify which affordances and permissions should be prioritized, such as structured threat modelling, pre-deployment agentic evaluations, post-deployment continuous monitoring, and AI safety cases. This paper proposes a complementary and empirical methodology that leverages existing use-case-specific benchmarks: backchaining LoC mitigations from the errors an AI system makes on national security benchmarks. The approach proceeds in three steps and allows national security deployers to start building LoC mitigations today, from evidence they can generate themselves. First, deployers evaluate AI systems on mission-specific benchmarks approximating real use-cases. Second, deployers concentrate on the incorrect responses that the AI system provides to the benchmark questions, and backchain the affordances and permissions that would enable the AI system to cause downstream harm if it pursued the actions described in the incorrect answers. Third, deployers intervene selectively on those affordances and permissions, bottlenecking the paths to harm while preserving the AI system's ability to carry out the correct action. We illustrate this methodology through a demonstrative benchmark question on derivative security classification.