AI-ECG Detection of Structural Heart Disease in the Community Setting: Transportability and Spectrum Effects in the...

Artificial intelligence (AI) diagnostic models are typically developed in hospital-based populations enriched for disease prevalence and severity, where distinctions between health and disease are pronounced. When deployed in community-dwelling populations with lower prevalence and milder phenotypes, these distinctions may become less well-defined, raising questions about transportability and the impact of disease spectrum on model performance.

The purpose of this study was to evaluate how differences in disease spectrum influence the transportability of AI-based diagnostic models into community-dwelling populations, using artificial intelligence electrocardiogram analysis (AI-ECG) for structural heart disease (SHD) as a test case.

EchoNext is an AI-ECG model trained in a multicenter, hospital-based cohort to detect SHD. We evaluated its performance in PREVUE-VALVE (Age and Sex-Specific PREValence of AcqUirEd VALVular Heart DiseasE Study), a community-based study of individuals aged 65 to 85 years undergoing in-home ECG and transthoracic echocardiography. Performance was assessed using area under the receiver operating characteristic curve (AUC) and predictive value and compared with derivation and external hospital-based cohorts, using propensity matching to account for differences in disease prevalence and case mix. Subgroup analyses were performed in clinically relevant populations.

Among 3,000 PREVUE-VALVE participants, 2,402 met criteria for analysis. Compared with hospital-based cohorts, PREVUE-VALVE had lower SHD prevalence (8% vs 43%), less severe disease, and a shift in phenotype, including more moderate tricuspid regurgitation and less systolic heart failure. Consistent with these differences, model discrimination was lower in PREVUE-VALVE than in the hospital-based cohort (AUC: 71% [95% CI: 66%-76%] vs 83% [95% CI: 82%-83%]). Propensity matching attenuated but did not eliminate this difference, while performance was similar across external hospital-based cohorts, supporting disease spectrum and clinical context as key drivers. Performance was modestly better in PREVUE-VALVE subgroups with higher SHD prevalence and greater disease severity, such as individuals with an abnormal ECG (AUC: 79% [95% CI: 75%-83%]) or impaired health status (AUC: 76% [95% CI: 70%-82%]).

In a community-dwelling population with lower disease prevalence and milder phenotypes, AI-ECG performance was attenuated relative to hospital-based cohorts, driven by differences in disease spectrum. These findings underscore the impact of both disease prevalence and case mix on AI performance and highlight the importance of model evaluation within intended use populations. (Age and Sex-Specific PREValence if AcqUirEd VALVular Heart DiseasE; NCT05357404).