FTT-GRU: A Hybrid Fast Temporal Transformer with GRU for Remaining Useful Life Prediction

2025-11-01

AI Summary

Existing RUL prediction methods struggle to jointly capture global temporal dependencies and local degradation patterns from multivariate sensor data. This paper proposes FTT-GRU, a hybrid model that— for the first time—integrates a Fast Temporal Transformer (FTT) with FFT-accelerated linear attention and a GRU, enabling synergistic modeling of long-range dependencies and fine-grained degradation dynamics. Evaluated on CMAPSS FD001, it achieves RMSE=30.76, MAE=18.97, and R²=0.45, outperforming state-of-the-art deep baselines; its lightweight architecture incurs only 1.12 ms CPU inference latency per sample, ensuring training stability and industrial real-time deployability. Key contributions are: (i) an FFT-accelerated linear attention mechanism reducing Transformer computational complexity; (ii) the first FTT-GRU hybrid paradigm unifying global and local temporal modeling; and (iii) comprehensive validation of superior accuracy, efficiency, and practicality.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Accurate prediction of the remaining useful life (RUL) of industrial machinery is essential for reducing downtime and optimizing maintenance schedules. Existing approaches, such as long short-term memory (LSTM) networks and convolutional neural networks (CNNs), often struggle to model both global temporal dependencies and fine-grained degradation trends in multivariate sensor data. We propose a hybrid model, FTT-GRU, which combines a Fast Temporal Transformer (FTT) -- a lightweight Transformer variant using linearized attention via fast Fourier transform (FFT) -- with a gated recurrent unit (GRU) layer for sequential modeling. To the best of our knowledge, this is the first application of an FTT with a GRU for RUL prediction on NASA CMAPSS, enabling simultaneous capture of global and local degradation patterns in a compact architecture. On CMAPSS FD001, FTT-GRU attains RMSE 30.76, MAE 18.97, and $R^2=0.45$, with 1.12 ms CPU latency at batch=1. Relative to the best published deep baseline (TCN--Attention), it improves RMSE by 1.16% and MAE by 4.00%. Training curves averaged over $k=3$ runs show smooth convergence with narrow 95% confidence bands, and ablations (GRU-only, FTT-only) support the contribution of both components. These results demonstrate that a compact Transformer-RNN hybrid delivers accurate and efficient RUL predictions on CMAPSS, making it suitable for real-time industrial prognostics.

Problem

• Predicting remaining useful life of industrial machinery
• Modeling global and local degradation patterns in sensor data
• Improving prediction accuracy with compact hybrid architecture

Innovation

• Hybrid Fast Temporal Transformer with GRU model
• Linearized attention via fast Fourier transform
• Combining global and local degradation pattern capture