A Procedural Framework for Assessing the Desirability of Process Deviations

2025-06-13

AI Summary

Existing process conformance checking techniques identify deviations between process executions and models but cannot assess their desirability—i.e., whether they are problematic, acceptable, or beneficial—leading to subjective, inefficient, and non-reproducible manual evaluation. To address this gap, we propose the first structured, reproducible framework for assessing deviation desirability. Grounded in a systematic literature review and semi-structured expert interviews, the framework defines three mutually exclusive desirability categories—problematic, acceptable, and beneficial—each accompanied by actionable recommendations that integrate theoretical conceptualization with frontline practical insights. We empirically validate the framework through task-oriented experiments, demonstrating significant improvements in analysts’ assessment efficiency and inter-rater consistency. Crucially, it maintains comprehensiveness while supporting concise, actionable decision-making. This work provides a methodological foundation for evidence-based process deviation governance.

Technology Category

• Knowledge Representation and Reasoning: Preferences
• Search and Optimization: Evaluation and Analysis
• Data Mining & Knowledge Management: Anomaly/Outlier Detection

Application Category

• User Modeling, Personalization and Recommendation: Metrics for user behavior and evaluating success
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Responsible Web: Consent frameworks and practices on the web

Abstract

Conformance checking techniques help process analysts to identify where and how process executions deviate from a process model. However, they cannot determine the desirability of these deviations, i.e., whether they are problematic, acceptable or even beneficial for the process. Such desirability assessments are crucial to derive actions, but process analysts typically conduct them in a manual, ad-hoc way, which can be time-consuming, subjective, and irreplicable. To address this problem, this paper presents a procedural framework to guide process analysts in systematically assessing deviation desirability. It provides a step-by-step approach for identifying which input factors to consider in what order to categorize deviations into mutually exclusive desirability categories, each linked to action recommendations. The framework is based on a review and conceptualization of existing literature on deviation desirability, which is complemented by empirical insights from interviews with process analysis practitioners and researchers. We evaluate the framework through a desirability assessment task conducted with practitioners, indicating that the framework effectively enables them to streamline the assessment for a thorough yet concise evaluation.

Problem

• Assessing desirability of process deviations systematically
• Providing step-by-step framework for deviation categorization
• Streamlining manual, subjective desirability evaluations

Innovation

• Procedural framework for deviation desirability assessment
• Step-by-step approach with input factors
• Empirical insights from interviews and literature