On the Tunability of Random Survival Forests Model for Predictive Maintenance

2025-04-20

AI Summary

This paper addresses the instability of failure-time estimation by Random Survival Forests (RSF) in predictive maintenance, stemming from high sensitivity to hyperparameters. We propose the first three-tiered adjustability quantification framework—operating at the model, hyperparameter, and value-range levels—to systematically assess the impact of each hyperparameter on discriminative performance (C-index) and calibration accuracy (Brier score). Our analysis reveals, for the first time, that the *splitrule* exhibits negative adjustability: improper tuning degrades performance. Evaluated on all four C-MAPSS aircraft engine degradation subsets, optimized RSF achieves an average C-index improvement of 0.0547 and an average Brier score reduction of 0.0199. Hyperparameters *ntree* and *mtry* demonstrate the strongest adjustability, while *nodesize* proves robust within the range [10, 30]. The framework provides an interpretable, empirically grounded basis for reliable RSF deployment in industrial prognostics.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Search and Optimization: Metareasoning and Metaheuristics
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

This paper investigates the tunability of the Random Survival Forest (RSF) model in predictive maintenance, where accurate time-to-failure estimation is crucial. Although RSF is widely used due to its flexibility and ability to handle censored data, its performance is sensitive to hyperparameter configurations. However, systematic evaluations of RSF tunability remain limited, especially in predictive maintenance contexts. We introduce a three-level framework to quantify tunability: (1) a model-level metric measuring overall performance gain from tuning, (2) a hyperparameter-level metric assessing individual contributions, and (3) identification of optimal tuning ranges. These metrics are evaluated across multiple datasets using survival-specific criteria: the C-index for discrimination and the Brier score for calibration. Experiments on four CMAPSS dataset subsets, simulating aircraft engine degradation, reveal that hyperparameter tuning consistently improves model performance. On average, the C-index increased by 0.0547, while the Brier score decreased by 0.0199. These gains were consistent across all subsets. Moreover, ntree and mtry showed the highest average tunability, while nodesize offered stable improvements within the range of 10 to 30. In contrast, splitrule demonstrated negative tunability on average, indicating that improper tuning may reduce model performance. Our findings emphasize the practical importance of hyperparameter tuning in survival models and provide actionable insights for optimizing RSF in real-world predictive maintenance applications.

Problem

• Evaluating RSF hyperparameter tunability for predictive maintenance
• Quantifying tuning impact on time-to-failure estimation accuracy
• Identifying optimal hyperparameter ranges for RSF performance

Innovation

• Three-level framework for RSF tunability evaluation
• Survival-specific metrics: C-index and Brier score
• Identified optimal ranges for key hyperparameters