PerFormer: A Permutation Based Vision Transformer for Remaining Useful Life Prediction

2025-05-30

AI Summary

Multivariate time-series data lack inherent spatial structure, hindering direct application of Vision Transformers (ViTs) to Remaining Useful Life (RUL) prediction. To address this, we propose a temporal rearrangement encoding framework that maps sensor time-series into image-like representations via a learnable permutation matrix, optimized end-to-end using a novel permutation loss. This work is the first to systematically adapt the ViT architecture to the Prognostics and Health Management (PHM) domain, overcoming conventional time-series models’ reliance on local dependencies or predefined structural assumptions. Evaluated on the NASA C-MAPSS dataset, our method significantly outperforms CNN-, RNN-, and Transformer-based baselines, achieving a 12.7% reduction in mean absolute RUL prediction error and establishing new state-of-the-art performance. The results demonstrate both the effectiveness and practical applicability of our approach for industrial-scale predictive maintenance.

Technology Category

• Computer Vision: Multi-modal Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Accurately estimating the remaining useful life (RUL) for degradation systems is crucial in modern prognostic and health management (PHM). Convolutional Neural Networks (CNNs), initially developed for tasks like image and video recognition, have proven highly effectively in RUL prediction, demonstrating remarkable performance. However, with the emergence of the Vision Transformer (ViT), a Transformer model tailored for computer vision tasks such as image classification, and its demonstrated superiority over CNNs, there is a natural inclination to explore its potential in enhancing RUL prediction accuracy. Nonetheless, applying ViT directly to multivariate sensor data for RUL prediction poses challenges, primarily due to the ambiguous nature of spatial information in time series data. To address this issue, we introduce the PerFormer, a permutation-based vision transformer approach designed to permute multivariate time series data, mimicking spatial characteristics akin to image data, thereby making it suitable for ViT. To generate the desired permutation matrix, we introduce a novel permutation loss function aimed at guiding the convergence of any matrix towards a permutation matrix. Our experiments on NASA's C-MAPSS dataset demonstrate the PerFormer's superior performance in RUL prediction compared to state-of-the-art methods employing CNNs, Recurrent Neural Networks (RNNs), and various Transformer models. This underscores its effectiveness and potential in PHM applications.

Problem

• Enhancing RUL prediction accuracy using Vision Transformers
• Addressing challenges in applying ViT to multivariate sensor data
• Introducing PerFormer for improved performance in PHM applications

Innovation

• Permutation-based Vision Transformer for RUL
• Novel permutation loss function for matrices
• Superior performance on C-MAPSS dataset