Structured-to-text ClinicalBERT embeddings with random Forest for heart disease prediction: a proof-of-concept study...

Heart disease remains one of the leading causes of mortality worldwide, highlighting the need for accurate and early risk prediction systems. Traditional machine learning approaches for cardiovascular disease prediction primarily rely on structured clinical attributes and may not fully capture contextual relationships among patient features. To address this limitation, this study proposes a structured-to-text ClinicalBERT framework that transforms structured cardiovascular records into contextual clinical text representations and utilizes transformer-based embeddings for heart disease prediction. The study employs a publicly available UCI Statlog/Kaggle heart disease dataset containing 270 complete patient records. Structured cardiovascular attributes, including age, sex, chest pain type, blood pressure, cholesterol level, electrocardiogram results, and heart rate measurements, are converted into clinically meaningful textual descriptions. These text representations are processed using ClinicalBERT to generate contextual embeddings, which are subsequently used as input features for a Random Forest classifier. Model performance was evaluated using an 80:20 train-test split and assessed through Accuracy, Precision, Recall, F1-score, and ROC-AUC metrics. Experimental results demonstrate that the proposed ClinicalBERT + Random Forest framework achieved an accuracy of 95.6%, precision of 88.89%, recall of 95.30%, F1-score of 91.30%, and a ROC-AUC of 0.71 on the held-out test set. Comparative analysis with conventional machine learning models indicates that contextual embeddings generated by ClinicalBERT provide improved feature representation for cardiovascular risk prediction. The findings demonstrate the feasibility of adapting ClinicalBERT to structured cardiovascular data through contextual text generation. Although the proposed framework shows promising predictive performance, the study should be considered a proof-of-concept due to the limited dataset size and absence of external validation. Future work will focus on multicenter evaluation, explainable AI techniques, and broader clinical validation to enhance generalizability and real-world applicability.