CORTEX: Composite Overlay for Risk Tiering and Exposure in Operational AI Systems

2025-08-24

AI Summary

Frequent failures of AI systems in high-risk domains pose urgent real-world safety challenges. Method: This paper introduces the first five-layer dynamic risk scoring framework integrating technical vulnerabilities, regulatory compliance, and deployment context. Grounded in analysis of over 1,200 AI incidents, it identifies 29 critical vulnerability categories and innovatively combines a likelihood-impact model, governance-and-context overlay mechanisms, quantified technical attack surface, and environment-specific modulation factors. Risk modeling employs Bayesian aggregation and Monte Carlo simulation to robustly capture long-tail and volatile risks. Contribution/Results: The framework generates actionable composite risk scores, enabling practical applications including AI risk registries, model audits, compliance assessments, and dynamic governance dashboards—thereby significantly enhancing the assessability and governability of AI systems in high-stakes operational environments.

Technology Category

• Philosophy and Ethics of AI: Safety, Robustness & Trustworthiness
• Natural Language Processing: Safety and Robustness
• Humans and AI: Planning and Decision Support for Human-Machine Teams

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Security and Privacy: Security and privacy of machine learning and AI applications

Abstract

As the deployment of Artificial Intelligence (AI) systems in high-stakes sectors - like healthcare, finance, education, justice, and infrastructure has increased - the possibility and impact of failures of these systems have significantly evolved from being a theoretical possibility to practical recurring, systemic risk. This paper introduces CORTEX (Composite Overlay for Risk Tiering and Exposure), a multi-layered risk scoring framework proposed to assess and score AI system vulnerabilities, developed on empirical analysis of over 1,200 incidents documented in the AI Incident Database (AIID), CORTEX categorizes failure modes into 29 technical vulnerability groups. Each vulnerability is scored through a five-tier architecture that combines: (1) utility-adjusted Likelihood x Impact calculations; (2) governance + contextual overlays aligned with regulatory frameworks, such as the EU AI Act, NIST RMF, OECD principles; (3) technical surface scores, covering exposure vectors like drift, traceability, and adversarial risk; (4) environmental and residual modifiers tailored to context of where these systems are being deployed to use; and (5) a final layered assessment via Bayesian risk aggregation and Monte Carlo simulation to model volatility and long-tail risks. The resulting composite score can be operationalized across AI risk registers, model audits, conformity checks, and dynamic governance dashboards.

Problem

• Assessing AI system vulnerabilities in high-stakes sectors
• Categorizing failure modes into technical vulnerability groups
• Developing multi-layered risk scoring framework for AI systems

Innovation

• Multi-layered risk scoring framework for AI vulnerabilities
• Five-tier architecture combining likelihood, impact and governance
• Bayesian risk aggregation with Monte Carlo simulation modeling