SAIF: A Comprehensive Framework for Evaluating the Risks of Generative AI in the Public Sector

2025-01-15

AI Summary

This study addresses bias, hallucination, jailbreaking, and regulatory compliance risks arising from generative AI deployment in public administration (e.g., welfare distribution, immigration adjudication). Methodologically, it introduces the first unified, multimodal AI risk taxonomy co-developed from governmental policies and industry guidelines; and proposes a scalable, four-stage evaluation framework—risk decomposition, scenario modeling, jailbreak injection, and prompt-type exploration—to dynamically accommodate emerging adversarial techniques and prompting paradigms. The key contributions include standardized, reproducible generation of risk-oriented prompt datasets, substantially improving assessment coverage and operational responsiveness. The framework provides a theoretically rigorous yet operationally feasible methodology for AI governance in the public sector, bridging policy intent with technical implementation in high-stakes administrative contexts.

Technology Category

• Philosophy and Ethics of AI: Safety, Robustness & Trustworthiness
• Humans and AI: AI for Accessibility
• Natural Language Processing: Safety and Robustness

Application Category

• Social Networks and Social Media: Generative AI / large language models and their impact on social systems
• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

The rapid adoption of generative AI in the public sector, encompassing diverse applications ranging from automated public assistance to welfare services and immigration processes, highlights its transformative potential while underscoring the pressing need for thorough risk assessments. Despite its growing presence, evaluations of risks associated with AI-driven systems in the public sector remain insufficiently explored. Building upon an established taxonomy of AI risks derived from diverse government policies and corporate guidelines, we investigate the critical risks posed by generative AI in the public sector while extending the scope to account for its multimodal capabilities. In addition, we propose a Systematic dAta generatIon Framework for evaluating the risks of generative AI (SAIF). SAIF involves four key stages: breaking down risks, designing scenarios, applying jailbreak methods, and exploring prompt types. It ensures the systematic and consistent generation of prompt data, facilitating a comprehensive evaluation while providing a solid foundation for mitigating the risks. Furthermore, SAIF is designed to accommodate emerging jailbreak methods and evolving prompt types, thereby enabling effective responses to unforeseen risk scenarios. We believe that this study can play a crucial role in fostering the safe and responsible integration of generative AI into the public sector.

Problem

• AI Risk Assessment
• Public Sector
• Responsible AI

Innovation

• SAIF framework
• Generative AI risk assessment
• Responsible AI application