A transformer model with enhanced feature learning and its application in rotating machinery diagnosis

Deep learning has become the prevailing trend of intelligent fault diagnosis for rotating machines. Compared to early-stage methods, deep learning methods use automatic feature extraction instead of manual feature design. However, conventional intelligent diagnosis models are trapped by a dilemma that simple models are unable to tackle difficult cases, while complicated models are likely to over-parameterize. In this paper, a transformer-based model, Periodic Representations for Transformers (PRT) is proposed. PRT uses a dense-overlapping split strategy to enhance the feature learning inside sequence patches. Combined with the inherent capability of capturing long range dependencies of transformer, and the further information extraction of class-attention, PRT has excellent feature extraction abilities and could capture characteristic features directly from raw vibration signals. Moreover, PRT adopts a two-stage positional encoding method to encode position information both among and inside patches, which could adapt to different input lengths. A novel inference method to use larger inference sample sizes is further proposed to improve the performance of PRT. The effectiveness of PRT is verified on two datasets, where it achieves comparable and even better accuracies than the benchmark and state-of-the-art methods. PRT has the least FLOPs among the best performing models and could be further improved by the inference strategy, reaching an accuracy near 100%.