Automatic Association of Quality Requirements and Quantifiable Metrics for Cloud Security Certification

2025-03-12

AI Summary

To address the low efficiency and high cost of manual mapping between natural-language security requirements and quantifiable evaluation metrics in the European Union Cloud Security (EUCS) certification framework, this paper introduces Sentence Transformers to the requirement–metric matching task for the first time. We propose an end-to-end semantic matching model that leverages sentence embeddings and similarity-based ranking to automatically associate all 70 EUCS requirements with 163 evaluation metrics. Experimental results demonstrate a Normalized Discounted Cumulative Gain (NDCG@10) of 0.640—surpassing prior approaches by 0.146—and substantially reduce reliance on manual effort. This work establishes a reusable methodological framework and empirical foundation for automating cloud security certification assessments, enabling scalable, consistent, and auditable alignment between regulatory requirements and measurable technical controls.

Technology Category

• Natural Language Processing: Safety and Robustness
• Machine Learning: Calibration & Uncertainty Quantification
• Constraint Satisfaction and Optimization: Satisfiability Modulo Theories

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Security and Privacy: Large-scale security measurements
• User Modeling, Personalization and Recommendation: Metrics for user behavior and evaluating success

Abstract

The European Cybersecurity Certification Scheme for Cloud Services (EUCS) is one of the first cybersecurity schemes in Europe, defined by the European Union Agency for Cybersecurity (ENISA). It aims to encourage cloud providers to strengthen their cybersecurity policies in order to receive an official seal of approval from European authorities. EUCS defines a set of security requirements that the cloud provider must meet, in whole or in part, in order to achieve the security certification. The requirements are written in natural language and cover every aspect of security in the cloud environment, from logging access to protecting the system with anti-malware tools to training staff. Operationally, each requirement is associated with one or more evaluable metrics. For example, a requirement to monitor access attempts to a service will have associated metrics that take into account the number of accesses, the number of access attempts, who is accessing, and what resources are being used. Partners in the European project Medina, which ended in October 2023, defined 163 metrics and manually mapped them to 70 EUCS requirements. Manual mapping is intuitively a long and costly process in terms of human resources. This paper proposes an approach based on Sentence Transformers to automatically associate requirements and metrics. In terms of correctness of associations, the proposed method achieves a Normalized Discounted Cumulative Gain of 0.640, improving a previous experiment by 0.146 points.

Problem

• Automate mapping of cloud security requirements to metrics.
• Improve efficiency in EUCS certification process.
• Enhance accuracy in associating security metrics with requirements.

Innovation

• Automated mapping using Sentence Transformers
• Improved association accuracy by 0.146 points
• Linked 163 metrics to 70 EUCS requirements