Related Experiment Video
Updated: Jul 6, 2025

A Swin Transformer-Based Model for Thyroid Nodule Detection in Ultrasound Images
Published on: April 21, 2023
Enhancing heart disease prediction using a self-attention-based transformer model
Atta Ur Rahman1,2, Yousef Alsenani3,4, Adeel Zafar5
1Riphah Institute of System Engineering, Riphah International University Islamabad, Islamabad, 46000, Pakistan. atta.rahman@riphah.edu.pk.
Insights
This study introduces a novel self-attention transformer model for early cardiovascular disease (CVD) risk prediction. The model achieved 96.51% accuracy, outperforming existing methods for heart disease detection.
Area of Science:
- Artificial Intelligence in Healthcare
- Machine Learning for Disease Prediction
- Biomedical Informatics
Background:
- Cardiovascular diseases (CVDs) are a leading global cause of mortality, necessitating accurate early detection methods.
- Current diagnostic approaches require timely and precise identification of heart failure risk factors.
- Automated systems analyzing patient characteristics can aid in early CVD diagnosis.
Purpose of the Study:
- To develop and evaluate a novel self-attention-based transformer model for predicting cardiovascular disease (CVD) risk.
- To enhance the accuracy and interpretability of automated heart disease prediction systems.
- To provide physicians with insights into the features driving model predictions for better clinical understanding.
Main Methods:
- Deployment of a novel self-attention-based transformer model integrating self-attention mechanisms and transformer networks.
- Utilizing self-attention layers to capture contextual information and model complex data patterns.
- Testing the model on the Cleveland dataset from the UCI machine learning repository.
Main Results:
- The proposed model achieved a highest accuracy of 96.51% on the Cleveland dataset.
- The model demonstrated superior performance compared to several baseline approaches.
- Experimental outcomes indicate a higher prediction rate than other state-of-the-art methods for heart disease prediction.
Conclusions:
- The self-attention-based transformer model offers a highly accurate and interpretable solution for early CVD risk prediction.
- This approach has the potential to significantly improve clinical trial efficacy and patient therapy.
- The model's ability to identify key predictive features aids physician comprehension and trust in automated diagnostics.
Abstract:
Cardiovascular diseases (CVDs) continue to be the leading cause of more than 17 million mortalities worldwide. The early detection of heart failure with high accuracy is crucial for clinical trials and therapy. Patients will be categorized into various types of heart disease based on characteristics like blood pressure, cholesterol levels, heart rate, and other characteristics. With the use of an automatic system, we can provide early diagnoses for those who are prone to heart failure by analyzing their characteristics. In this work, we deploy a novel self-attention-based transformer model, that combines self-attention mechanisms and transformer networks to predict CVD risk. The self-attention layers capture contextual information and generate representations that effectively model complex patterns in the data. Self-attention mechanisms provide interpretability by giving each component of the input sequence a certain amount of attention weight. This includes adjusting the input and output layers, incorporating more layers, and modifying the attention processes to collect relevant information. This also makes it possible for physicians to comprehend which features of the data contributed to the model's predictions. The proposed model is tested on the Cleveland dataset, a benchmark dataset of the University of California Irvine (UCI) machine learning (ML) repository. Comparing the proposed model to several baseline approaches, we achieved the highest accuracy of 96.51%. Furthermore, the outcomes of our experiments demonstrate that the prediction rate of our model is higher than that of other cutting-edge approaches used for heart disease prediction.

