Multimodal heart failure prediction model based on graph convolutional neural network

Heart failure (HF) is a common and life-threatening cardiovascular disease, and early accurate diagnosis is critical for improving patient survival rates and optimizing treatment outcomes. This work integrated the electrocardiogram (ECG) signals of patients with clinical text data to construct a 12-lead ECG-Text-LVEF Cardio dataset. A multimodal HF prediction model based on graph convolutional neural network (GCN) was proposed in this work. The model employs global feature vectors extracted from patients' ECG spectrograms as node features, generates medically semantic vectors by encoding clinical text via ClinicalBERT, establishes connections between nodes that meet a preset threshold after calculating cosine similarity between these vectors, and thus constructs a semantic adjacency graph that quantifies similarities in clinical descriptions among patients and explicitly models common associations in their symptom presentations and disease characteristics. Finally, it achieves reasoning from individual features to group relationships through 2-layer graph modeling. At the same time, the Top-K adjacency strategy and similarity threshold control the sparsity of edge connections, and the edge perturbation mechanism is introduced to enhance the generalization ability of the model. Experimental results demonstrate that the model achieves an accuracy of 98.5%, a precision of 98.4%, a recall of 99%, and an AUC of 0.997, validating the effectiveness of multimodal feature fusion in HF prediction and providing a practical pathway for applying GCNs to multimodal medical risk prediction tasks.

The online version contains supplementary material available at 10.1007/s13534-026-00560-5.