An LLM-based Two-Stage Transformer Framework for Cross-Domain Bearing Fault Diagnosis with Limited Data

2026-06-23

AI Summary

This study addresses the challenge of cross-domain bearing fault diagnosis in industrial settings, where data heterogeneity, varying operating conditions, and scarce labeled data in the target domain hinder performance. To tackle this, the authors propose a knowledge-guided two-stage transfer learning framework. The method employs a lightweight causal self-attention Transformer to hierarchically extract vibration features and explicitly constructs knowledge transfer pathways through multi-source pretraining, fault prototype embedding, and a knowledge modulation mechanism. Furthermore, a class-adaptive classifier is introduced to mitigate feature distribution shifts and label space heterogeneity. Evaluated on four real-world datasets, the approach achieves an average accuracy of 92.61% using only 10% of labeled target data, outperforming the current state-of-the-art by 17.24 percentage points.

Abstract

Bearing fault diagnosis faces critical challenges when dataset heterogeneity, operating condition variations, and limited labeled data occur simultaneously in industrial environments. Existing approaches address these issues in isolation and rely on implicit feature alignment, limiting effectiveness under concurrent challenges. This paper proposes a knowledge-guided two-stage transfer learning framework that employs a lightweight GPT-2-style Transformer with causal self-attention for hierarchical feature extraction from vibration signals, establishing explicit pathways where pre-trained encoder weights and fault prototype embeddings serve as knowledge carriers from multi-source pre-training to target adaptation. The framework addresses the dual-shift challenge through multi-source learning for generalizable representations, prototype-based knowledge modulation for target adaptation, and taxonomy-adaptive classification for seamless transfer across heterogeneous fault categories. Experimental validation on four real-world datasets demonstrates 92.61% average accuracy with only 10% labeled target data, outperforming state-of-the-art methods by 17.24 percentage points, establishing a practical pathway toward cost-effective predictive maintenance in Industry 4.0 applications.

Problem

• bearing fault diagnosis
• cross-domain
• limited labeled data
• dataset heterogeneity
• operating condition variations

Innovation

• two-stage transfer learning
• prototype-based knowledge modulation
• causal self-attention Transformer
• cross-domain fault diagnosis
• limited labeled data