Predictive Multiplicity in Survival Models: A Method for Quantifying Model Uncertainty in Predictive Maintenance Applications

2025-04-16

AI Summary

In survival analysis, multiple high-accuracy models often yield substantially divergent predictions—termed “predictive multiplicity”—regarding failure times and risk scores for the same equipment, undermining decision reliability in high-stakes scenarios. This work is the first to systematically define and quantify predictive multiplicity within a survival analysis framework, introducing a three-dimensional metric suite tailored to censored data: ambiguity (rate of prediction inconsistency), discrepancy (divergence between survival functions), and obscurity (intensity of uncertainty perception). Methodologically, we integrate multi-model ensembling, censoring-aware uncertainty estimation, and survival-function divergence measurement, validated on real-world industrial datasets including C-MAPSS. Experiments reveal that up to 40–45% of observations exhibit significant ambiguity, highlighting non-negligible inter-model disagreement at the individual level. Our framework bridges a critical theoretical gap and delivers interpretable, quantifiable uncertainty measures to enhance reliability in predictive maintenance.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Uncertainty Representations
• Multiagent Systems: Multiagent Systems under Uncertainty

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for heterogeneous, signed, attributed, multi-relational, temporal, higher-order, and annotated Web-related graphs
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Web Mining and Content Analysis: Web data provenance, reliability, and authenticity

Abstract

In many applications, especially those involving prediction, models may yield near-optimal performance yet significantly disagree on individual-level outcomes. This phenomenon, known as predictive multiplicity, has been formally defined in binary, probabilistic, and multi-target classification, and undermines the reliability of predictive systems. However, its implications remain unexplored in the context of survival analysis, which involves estimating the time until a failure or similar event while properly handling censored data. We frame predictive multiplicity as a critical concern in survival-based models and introduce formal measures -- ambiguity, discrepancy, and obscurity -- to quantify it. This is particularly relevant for downstream tasks such as maintenance scheduling, where precise individual risk estimates are essential. Understanding and reporting predictive multiplicity helps build trust in models deployed in high-stakes environments. We apply our methodology to benchmark datasets from predictive maintenance, extending the notion of multiplicity to survival models. Our findings show that ambiguity steadily increases, reaching up to 40-45% of observations; discrepancy is lower but exhibits a similar trend; and obscurity remains mild and concentrated in a few models. These results demonstrate that multiple accurate survival models may yield conflicting estimations of failure risk and degradation progression for the same equipment. This highlights the need to explicitly measure and communicate predictive multiplicity to ensure reliable decision-making in process health management.

Problem

• Quantifying predictive multiplicity in survival models for reliability
• Measuring ambiguity, discrepancy, obscurity in predictive maintenance
• Addressing conflicting risk estimates in survival-based model outputs

Innovation

• Quantifying predictive multiplicity in survival models
• Introducing ambiguity, discrepancy, obscurity measures
• Applying methodology to predictive maintenance datasets