Transformer-based temporal sequence learners for arrhythmia classification

An electrocardiogram (ECG) plays a crucial role in identifying and classifying cardiac arrhythmia. Traditional methods employ handcrafted features, and more recently, deep learning methods use convolution and recursive structures to classify heart signals. Considering the time sequence nature of the ECG signal, a transformer-based model with its high parallelism is proposed to classify ECG arrhythmia. The DistilBERT transformer model, pre-trained for natural language processing tasks, is used in the proposed work. The signals are denoised and then segmented around the R peak and oversampled to get a balanced dataset. The input embedding step is skipped, and only positional encoding is done. The final probabilities are obtained by adding a classification head to the transformer encoder output. The experiments on the MIT-BIH dataset show that the suggested model is excellent in classifying various arrhythmias. The model achieved 99.92% accuracy, 0.99 precision, sensitivity, and F1 score on the augmented dataset with a ROC-AUC score of 0.999.