Bionic Wearable ECG with Multimodal Large Language Models: Coherent Temporal Modeling for Early Ischemia Warning and...

Myocardial ischemia remains one of the principal causes of mortality and morbidity worldwide, necessitating novel approaches to facilitate early diagnosis and subsequent risk evaluation following reperfusion. Although advancements in wearables capable of ECG (electrocardiogram) monitoring have been initiated, these devices have encountered barriers due to limited capacities to encapsulate the temporally complex nature of ischemic events, notably in risk stratifying reperfusion injury. In this paper, we describe a framework that leverages bionic, wearable ECG sensor technologies along with multimodal large language models using a coherent temporal modeling effort to address the intertwining of fine-grained temporal dependencies, heterogeneous biomedical modalities, and interpretable risk stratification. Our temporally hierarchical fusion transformer utilizes a cross-granularity attention mechanism to model intrabeat, interbeat, and long-term dependencies all simultaneously. The validation of our system was carried out using 4 datasets across n = 108,778 patients, 17,173 of whom were ischemia-positive cases (4,627 from PTB-XL, 5,243 from MIMIC-IV, 6,891 from CODE-15%, and 412 in the wearable cohort). The area under receiver operating characteristic curve (AUROC) for the model for ischemia was 0.947, and the C-index for post-reperfusion risk stratification was 0.923, with a relative AUROC improvement of 4.8% to 9.5% over the best baseline in each dataset. Importantly, we achieved an average lead time of 18.4 min prior to the ischemic event to allow the clinician to enact interventions. Ultimately, this research demonstrates a prototype of an intelligent cardiovascular care monitoring system that couples advanced sensing with clinical decision support.