Fault Detection of Cyber-Physical Systems Using a Transfer Learning Method Based on Pre-Trained Transformers

Area of Science:

• Engineering
• Computer Science
• Artificial Intelligence

Background:

• Cyber-physical systems (CPSs) are crucial for modern industries, but failures can cause major disruptions.
• Prognostics and Health Management (PHM) is essential for CPS reliability.
• Deep learning (DL) methods face challenges with imbalanced and scarce fault-labeled data in CPS applications.

Purpose of the Study:

• To propose a transfer learning approach using a pre-trained transformer to improve fault detection in CPSs.
• To address the domain shift challenge between laboratory and real-world industrial data.
• To enhance the deployment of DL in industrial CPS settings with limited fault data.

Main Methods:

• Pre-training a streamlined transformer model on a large source dataset.
• Fine-tuning the model end-to-end on a smaller target dataset with a different distribution.
• Applying SHAP (SHapley Additive exPlanations) for explainable AI (XAI) to understand model decisions.

Main Results:

• The proposed transfer learning model achieved an average F1-score of 93.38% on a water purification facility dataset.
• The transformer-based approach outperformed baseline models like Convolutional Neural Networks (CNN) and Long Short-Term Memory (LSTM).
• Demonstrated practical effectiveness in industrial CPS settings with limited fault data.

Conclusions:

• Transformer-based transfer learning is a viable solution for enhancing fault detection in industrial CPSs.
• The method effectively transfers diagnostic knowledge, mitigating issues with data scarcity and domain shift.
• Explainable AI (XAI) methods like SHAP can increase the transparency of fault detection models.