Learning from Change: Predictive Models for Incident Prevention in a Regulated IT Environment

2026-04-15

AI Summary

This study addresses the challenge of proactive incident prevention in highly regulated IT environments by proposing an interpretable machine learning approach to predict the risk of incidents prior to change deployment. The method integrates organizational context features and leverages gradient-boosting models such as LightGBM, enhanced with SHAP values to provide transparent and auditable decision rationales. Experimental results demonstrate that the optimized LightGBM model significantly outperforms conventional rule-based systems, achieving a balance between high predictive performance and reliability while satisfying stringent audit and compliance requirements. These findings validate the feasibility of data-driven approaches in simultaneously meeting operational effectiveness and regulatory adherence objectives.

Technology Category

• Machine Learning: Transparent, Interpretable, Explainable ML
• Data Mining & Knowledge Management: Representing, Reasoning, and Using Provenance, Trust
• Natural Language Processing: Safety and Robustness

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Effective IT change management is important for businesses that depend on software and services, particularly in highly regulated sectors such as finance, where operational reliability, auditability, and explainability are essential. A significant portion of IT incidents are caused by changes, making it important to identify high-risk changes before deployment. This study presents a predictive incident risk scoring approach at a large international bank. The approach supports engineers during the assessment and planning phases of change deployments by predicting the potential of inducing incidents. To satisfy regulatory constraints, we built the model with auditability and explainability in mind, applying SHAP values to provide feature-level insights and ensure decisions are traceable and transparent. Using a one-year real-world dataset, we compare the existing rule-based process with three machine learning models: HGBC, LightGBM, and XGBoost. LightGBM achieved the best performance, particularly when enriched with aggregated team metrics that capture organisational context. Our results show that data-driven, interpretable models can outperform rule-based approaches while meeting compliance needs, enabling proactive risk mitigation and more reliable IT operations.

Problem

• incident prevention
• change management
• regulated IT environment
• risk prediction
• auditability

Innovation

• predictive risk scoring
• explainable AI
• SHAP
• change management
• regulated IT environment