Multi-Variable Transformer-Based Meta-Learning for Few-Shot Fault Diagnosis of Large-Scale Systems

Fault diagnosis in large-scale systems presents significant challenges due to the complexity and high dimensionality of data, as well as the scarcity of labeled fault data, which are hard to obtain during the practical operation process. This paper proposes a novel approach, called Multi-Variable Meta-Transformer (MVMT), to tackle these challenges. In order to deal with the multi-variable time series data, we modify the Transformer model, which is the currently most popular model on feature extraction of time series. To enable the Transformer model to simultaneously receive continuous and state inputs, we introduced feature layers before the encoder to better integrate the characteristics of both continuous and state variables. Then, we adopt the modified model as the base model for meta-learning-more specifically, the Model-Agnostic Meta-Learning (MAML) strategy. The proposed method leverages the power of Transformers for handling multi-variable time series data and employs meta-learning to enable few-shot learning capabilities. The case studies conducted on the Tennessee Eastman Process database and a Power-Supply System database demonstrate the exceptional performance of fault diagnosis in few-shot scenarios, whether based on continuous-only data or a combination of continuous and state variables.