impact assessment

Insufficient disclosure of safety evaluations by frontier AI companies undermines evidence-based regulation. This paper proposes the first mandatory two-stage (pre- and post-intervention) safety assessment disclosure framework, demonstrating that single-stage evaluations systematically misrepresent AI system safety. Through empirical analysis and gap diagnosis, we identify three critical industry shortcomings: incomplete evaluation coverage, unquantified intervention effects, and non-comparable results. To address these, we design a standardized evaluation protocol and define minimum transparency requirements for regulators, coupled with a phased mandatory disclosure mechanism. The framework delivers a verifiable, comparable chain of safety evidence—enabling a shift from “black-box compliance” to “evidence-driven regulation.” It provides policymakers with an actionable, scalable institutional pathway for AI governance.

AI safety evaluation lacks consensus standards, limiting its utility for governance and policy decisions. This paper introduces the first practical AI safety evaluation framework, systematically integrating threat modeling, assessment design, and validity validation. It formally defines three essential criteria for “useful” evaluations—risk alignment, reproducibility, and scalability—along with associated quantitative parameters. Innovatively distinguishing formal metrics from real-world risk coverage, the framework establishes an evolutionary paradigm—from isolated tests to modular, composable evaluation suites. It synergistically integrates red-teaming, evaluation validity analysis, and cybersecurity best practices to jointly optimize reliability, construct validity, and operational feasibility. The resulting safety evaluation guidelines have been adopted by industry stakeholders and policymaking bodies, demonstrably enhancing the interpretability of evaluation outcomes and their actionable support for risk-informed decision-making.

Rapid advancements in AI systems are outpacing the development of systematic, auditable methodologies for assessing their societal and ecological risks; current practices rely heavily on implicit assumptions and ad hoc testing. Method: We propose the first Probabilistic Risk Assessment (PRA) framework tailored to AI systems, integrating established PRA paradigms from nuclear and aerospace domains with AI-first principles. It comprises: (1) AI-specific hazard analysis; (2) bidirectional causal modeling—forward (capability deficiencies → harms) and backward (harm溯源 → capability vulnerabilities); and (3) scenario decomposition coupled with reference-scale-based uncertainty quantification. Contribution/Results: The framework enables explicit assumption tracking and absolute risk quantification. Implemented as an open-source, structured Risk Report Card tool, it delivers comparable, traceable, and quantified holistic risk estimates—supporting collaborative, high-reliability AI governance among developers, evaluators, and regulators.

This study addresses the challenge of optimizing resource allocation for multi-hazard risk mitigation in U.S. homeland security and emergency management. We propose an integer linear programming (ILP) decision-support model that integrates probabilistic risk assessment (PRA) with multi-criteria consequence quantification. Methodologically, the model innovatively fuses heterogeneous historical data and publicly available information to enable joint modeling across 16 hazard types and six consequence dimensions, while selecting optimal mitigation projects under budget constraints. It further introduces a sensitivity-driven framework that jointly optimizes robustness and cost-effectiveness. Applied empirically in Iowa, the model generates a high-value portfolio of 52 mitigation projects, achieving an average 37% reduction in expected risk. Multi-scenario sensitivity analysis confirms solution robustness. The approach provides a scalable, methodologically rigorous foundation for evidence-based resilience investment.

In early-stage collaborative robot task design, safety experts struggle to comprehend task logic, and risk assessment outcomes often lack practical implementability. Method: This paper proposes a model-driven risk assessment approach based on Behavior Trees (BTs)—the first application of BTs in risk assessment—enabling early risk identification, formal verification, and end-to-end traceability via visual modeling. Integrating Model-Driven Engineering (MDE) with Human Factors evaluation, the method was empirically validated by cross-functional practitioners from five industrial enterprises. Contribution/Results: The approach significantly improves risk identification completeness (+32%) and enhances collaboration efficiency between safety experts and development teams, reducing communication overhead by 41%. It establishes a novel, industrial-grade paradigm for trustworthy robotic systems that unifies modeling, analysis, and implementation within a single coherent framework.

This study addresses the challenge in regional risk assessment posed by missing asset attributes, which introduces unquantifiable uncertainty in exposure information and compromises risk estimation accuracy. For the first time, it systematically decomposes and quantifies the uncertainty arising specifically from probabilistic exposure representation, isolating it from total risk uncertainty to elucidate its generation and propagation mechanisms within the assessment workflow. Methodologically, the research integrates machine learning with engineering rule sets to impute missing data and constructs a high-resolution bridge exposure inventory. Uncertainty propagation is then analyzed through a combination of analytical methods and Monte Carlo simulations. The approach significantly enhances the transparency of exposure modeling and improves the reliability of regional-scale risk assessments.

This study addresses the growing complexity of cyber threats to hydropower systems driven by increasing software integration, which renders traditional risk assessment methods inadequate for identifying compound risks arising from coordinated attacks. The work proposes a novel two-stage framework that integrates HAZOP with Bow-Tie analysis for cybersecurity risk assessment in hydropower systems. By extending HAZOP deviation analysis, it identifies 18 classes of potential deviations and incorporates cyber-induced threat scenarios. Coupled with the Bow-Tie barrier model, the approach reveals that shared network infrastructure undermines defense-in-depth and enables adversaries to cooperatively trigger multiple deviations to bypass protective measures. The findings demonstrate that conventional assumptions of deviation independence and barrier isolation no longer hold under cyber attack conditions, and the proposed framework significantly enhances the identification and modeling of compound cyber risks.

This study addresses critical limitations in current AI risk assessment methodologies, which often misapply traditional safety frameworks, leading to incomplete risk coverage and erroneous safety conclusions due to terminological imprecision and inadequate contextual fit. To overcome these issues, this work proposes the first end-to-end risk assessment framework that extends the concept of Operational Design Domain (ODD)—originally developed for autonomous vehicles—to general-purpose AI systems. By explicitly defining the intended operating conditions of AI systems, the framework establishes well-demarcated safety boundaries. It integrates principles from systems safety engineering, cybersecurity, and AI governance, harmonizing technical, societal, and ethical dimensions into a unified, rigorous terminology and a structured evaluation process. The resulting methodology offers a comprehensive risk assessment tool applicable across diverse AI domains, significantly enhancing developers’ and auditors’ ability to understand, validate, and credibly substantiate safety claims—thereby avoiding misleading assurances of security.

This study addresses the lack of a systematic framework for identifying critical input variables and conducting sensitivity analysis under uncertainty in complex simulations, particularly in military decision-making contexts. The authors propose a unified sensitivity analysis framework that integrates local and global methods—including variance-based, derivative-based, screening, and uncertainty quantification techniques—and strategically maps these approaches to specific decision objectives such as factor prioritization, fixing, variance reduction, and mapping. Innovatively, the framework introduces a “sensitivity audit” mechanism to enhance traceability of model assumptions and promote responsible model usage. By providing a structured guide for high-dimensional, complex simulation systems, this work significantly improves model interpretability, transparency, and the credibility of decisions derived from such models.

This study addresses the lack of intuitive and interpretable methods for quantifying variable influence in existing regression models. To this end, the authors propose Impact Range Assessment (IRA), a novel approach that robustly measures and ranks predictor importance by evaluating the maximum potential change a predictor can induce in the response variable across its entire range of values, relative to the total variation observed in the response. Experiments on both synthetic linear and nonlinear datasets, as well as a real-world particulate matter prediction case, demonstrate that IRA effectively distinguishes relevant from irrelevant variables with consistent and reliable results. By providing a clear, quantitative interpretation of each variable’s contribution, IRA significantly enhances model transparency and trustworthiness.