Disclosure or Marketing? Analyzing the Efficacy of Vendor Self-reports for Vetting Public-sector AI

2026-04-01

AI Summary

Public sector actors increasingly rely on vendor-provided AI transparency documents, such as FactSheets, for accountability and risk assessment, yet their practical utility remains empirically unexamined. This study addresses this gap through semi-structured interviews and a systematic content analysis of FactSheets published by the GovAI Coalition, revealing for the first time that these documents function dually as both marketing instruments and disclosure mechanisms in practice. The findings indicate that while FactSheets alone are insufficient to support robust technical evaluation, they play a critical role in fostering trust, enabling stakeholder alignment, and sustaining ongoing governance dialogues. Building on these insights, the paper proposes reconceptualizing FactSheets not merely as static informational artifacts but as relational governance tools that facilitate dynamic, iterative engagement between public institutions and AI vendors.

Abstract

Documentation-based disclosure has become a central governance strategy for responsible AI, particularly in public-sector procurement. Tools such as model cards, datasheets, and AI FactSheets are increasingly expected to support accountability, risk assessment, and informed decision-making across organizational boundaries. Yet there is limited empirical evidence about how these artifacts are produced, interpreted, and used in practice. In this paper, we present a qualitative study of the GovAI Coalition FactSheet, a widely adopted transparency document designed to support AI procurement and governance in government contexts. Drawing on semi-structured interviews with vendors and public-sector practitioners, alongside a systematic analysis of completed FactSheets, we examine how FactSheets are used, what information they surface, and where they fall short. We find that FactSheets are asked to serve multiple and conflicting purposes simultaneously: showcasing vendor offerings, supporting evaluation and due diligence, and facilitating early-stage dialogue between vendors and agencies. These competing expectations, combined with the structural constraints of voluntary and public self-disclosure, limit the ability of FactSheets to function as standalone evaluation or risk-assessment tools. At the same time, our findings suggest that when understood as relational artifacts used to establish trust, shared understanding, and ongoing dialogue, FactSheets can help create conditions that support more meaningful disclosure and governance over time.

Problem

• AI governance
• vendor self-disclosure
• transparency artifacts
• public-sector procurement
• AI documentation

Innovation

• AI governance
• transparency artifacts
• relational artifacts
• public-sector AI procurement
• vendor self-disclosure