Partially Observable Markov Decision Process Framework for Operating Condition Optimization Using Real-Time Degradation Signals

2025-12-07

AI Summary

To address the challenge of jointly modeling and decision-making for multi-sensor real-time degradation signals in complex engineering systems, this paper proposes a maintenance optimization framework based on a Partially Observable Markov Decision Process (POMDP) that integrates hidden-state estimation with degradation constraints. It achieves, for the first time, adaptive capacity control and predictive maintenance co-optimization driven by heterogeneous degradation signals. Methodologically, the framework unifies real-time signal fusion, Hidden Markov Modeling, and dynamic programming to explicitly characterize the coupling between observation uncertainty and degradation evolution. Experiments on bearing and aircraft engine datasets demonstrate significant improvements: system reliability is enhanced, maintenance costs are reduced, and decision response speed increases by over 40%. These results validate the framework’s superior efficiency, robustness, and practicality.

Technology Category

• Planning, Routing, and Scheduling: Planning with Markov Models (MDPs, POMDPs)
• Search and Optimization: Mixed Discrete/Continuous Search
• Constraint Satisfaction and Optimization: Mixed Discrete/Continuous Optimization

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization

Abstract

In many engineering systems, proper predictive maintenance and operational control are essential to increase efficiency and reliability while reducing maintenance costs. However, one of the major challenges is that many sensors are used for system monitoring. Analyzing these sensors simultaneously for better predictive maintenance optimization is often very challenging. In this paper, we propose a systematic decision-making framework to improve the system performance in manufacturing practice, considering the real-time degradation signals generated by multiple sensors. Specifically, we propose a partially observed Markov decision process (POMDP) model to generate the optimal capacity and predictive maintenance policies, given the fact that the observation of the system state is imperfect. Such work provides a systematic approach that focuses on jointly controlling the operating conditions and preventive maintenance utilizing the real-time machine deterioration signals by incorporating the degradation constraint and non-observable states. We apply this technique to the bearing degradation data and NASA aircraft turbofan engine dataset, demonstrating the effectiveness of the proposed method.

Problem

• Optimizes operational control using real-time degradation signals
• Addresses imperfect system state observation via POMDP framework
• Integrates predictive maintenance with operating condition optimization

Innovation

• POMDP model optimizes capacity and maintenance policies
• Incorporates real-time degradation signals from multiple sensors
• Handles imperfect system state observations with degradation constraints