Silent Updates: Measuring and Closing the Post-Deployment Disclosure Gap

2026-08-12

AI Summary

This study addresses the widespread but previously unexamined practice of “silent updates”—undisclosed version changes to deployed foundation models—which undermines the verifiability and documentation consistency essential for effective AI governance. Introducing the concept of silent updates, this work quantifies their associated risks through empirical auditing of documentation-to-deployment alignment across nine major API providers and seven inference platforms. Combining policy review with systematic behavioral monitoring, the authors design a three-part behavioral trigger framework to delineate when re-disclosure obligations should apply and develop the first cross-platform transparency assessment tool, the Silent Update Scorecard. The analysis reveals that none of the examined services provide externally verifiable evidence linking deployed models to their documentation, exposing a systemic disclosure gap and offering empirical grounding for regulatory and standardization efforts.

Abstract

Deployed foundation models are often not static systems, with providers able to modify system behavior through fine-tuning, classifier updates, system prompt revisions, retrieval changes, and routing changes. These updates can be made silently -- that is, without public disclosure, a version increment, or re-evaluation. Such silent updates challenge a core assumption behind current AI governance frameworks that an externally verifiable chain of custody links the model referred to in evaluation results or a system card to the model served to users. In this paper, we examine post-deployment disclosure practices across first-party API providers and inference hosts to establish the extent to which a chain of custody exists in practice. We find that providers commonly publish substantial safety documentation, including quantitative evaluations and version-specific reports, but no provider in our sample published information allowing an external party to verify that the artifact being served is the same one referred to in this documentation. We introduce the Silent Updates Scorecard, a public instrument for measuring post-deployment disclosure practices across providers and hosts, and preliminary results for a sample of nine first-party API providers and seven third-party inference hosts. We also propose a Three-Part Behavioral Trigger System for determining when post-deployment modifications to a system motivate disclosure or re-evaluation obligations.

Problem

• silent updates
• foundation models
• post-deployment disclosure
• chain of custody
• AI governance

Innovation

• silent updates
• post-deployment disclosure
• chain of custody
• behavioral trigger system
• foundation model governance