Predicting LLM Safety Before Release by Simulating Deployment

2026-07-08

AI Summary

Current pre-deployment safety evaluations often fail to accurately predict the frequency of undesirable behaviors in large language models during real-world deployment due to insufficient coverage, unrepresentative samples, and susceptibility to being recognized by models as test inputs. This work proposes a deployment simulation method grounded in authentic dialogue prefixes: by fixing historical context and prompting candidate models to generate subsequent responses, it enables auditing of novel alignment failures and estimation of risk incidence rates. The approach leverages publicly available chat data to construct evaluation scenarios, allowing external researchers to conduct realistic safety assessments without access to proprietary logs. Prospective and retrospective experiments on the GPT-5 model series demonstrate that this method significantly outperforms baselines based on adversarial production data, yielding predictions that align more closely with observed misbehavior rates in actual deployment and proving feasible even in complex tool-use settings.

Abstract

Pre-deployment safety evaluations aim to inform the downstream risks of releasing a new AI model. Yet most evaluations provide limited evidence about how often undesired model behavior will occur in deployment: they generally have insufficient coverage, are unrepresentative, and are generally recognizable as tests. To address these concerns, we study a simple way to simulate a model deployment: starting from de-identified conversations from a previous model deployment, we hold fixed the initial conversation prefix and regenerate the next response using a candidate model. The resulting responses can then both be audited for novel misalignments and used to estimate the prevalence of model misbehavior before deployment. We evaluate deployment simulation across four GPT-5-series deployments, using registered, outcome-blinded predictions for GPT-5.4 and retrospective analyses of three earlier releases. We find that deployment simulation produces informative estimates of post-deployment misbehavior rates and outperforms baselines based on adversarially selected production data; its evaluation-awareness point estimates were also much closer to production traffic than those from traditional evaluations. We also identify the realism of tool resampling as a central challenge for further improving predictions and share results suggesting that this challenge is surmountable even in complex tool-use settings. Finally, we show that deployment simulation can be seeded from public chat datasets and remain informative about production misbehavior rates, suggesting a path for external researchers to run deployment-grounded evaluations without access to private production logs. Overall, deployment simulation helps evaluators forecast how language models will behave in the real world and supports more quantitative assessment of deployment risk.

Problem

• LLM safety
• pre-deployment evaluation
• deployment simulation
• model misbehavior
• risk prediction

Innovation

• deployment simulation
• pre-deployment evaluation
• LLM safety
• misbehavior prediction
• real-world alignment