Combining Cost-Constrained Runtime Monitors for AI Safety

2025-07-19

AI Summary

This paper addresses the problem of optimally composing multiple runtime monitors under an average cost constraint to maximize the safety intervention probability (i.e., recall) against AI misaligned outputs. The proposed method introduces a Neyman–Pearson lemma–based optimization framework that unifies monitor invocation timing, selection, and intervention decisions into a likelihood-ratio–driven sequential decision problem. Pareto-optimal solutions are identified via exhaustive search, enabling principled trade-offs between performance and computational cost. Empirical evaluation on code review tasks demonstrates that the approach significantly improves multi-monitor coordination efficiency, achieving over 100% recall improvement relative to baseline methods. These results validate both the theoretical soundness and practical efficacy of the framework in resource-constrained real-world deployment scenarios.

Technology Category

• Reasoning under Uncertainty: Sequential Decision Making
• Multiagent Systems: Mechanism Design
• Constraint Satisfaction and Optimization: Constraint Learning and Acquisition

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

Monitoring AIs at runtime can help us detect and stop harmful actions. In this paper, we study how to combine multiple runtime monitors into a single monitoring protocol. The protocol's objective is to maximize the probability of applying a safety intervention on misaligned outputs (i.e., maximize recall). Since running monitors and applying safety interventions are costly, the protocol also needs to adhere to an average-case budget constraint. Taking the monitors' performance and cost as given, we develop an algorithm to find the most efficient protocol. The algorithm exhaustively searches over when and which monitors to call, and allocates safety interventions based on the Neyman-Pearson lemma. By focusing on likelihood ratios and strategically trading off spending on monitors against spending on interventions, we more than double our recall rate compared to a naive baseline in a code review setting. We also show that combining two monitors can Pareto dominate using either monitor alone. Our framework provides a principled methodology for combining existing monitors to detect undesirable behavior in cost-sensitive settings.

Problem

• Combine multiple runtime monitors for AI safety efficiently
• Maximize recall of safety interventions under cost constraints
• Develop algorithm to optimize monitor and intervention allocation

Innovation

• Combines multiple runtime monitors into single protocol
• Uses Neyman-Pearson lemma for intervention allocation
• Optimizes recall under budget constraints strategically