Related Experiment Videos
Discordance between reported discrimination and translational-readiness evidence in AI-based ASCVD prediction: a
Ruifeng Liu1, Ross Arena2,3, Carl Lavie4
1Beijing Friendship Hospital, Capital Medical University, Beijing, China.
Importance:
The development of artificial intelligence (AI) models for atherosclerotic cardiovascular disease (ASCVD) risk prediction has rapidly accelerated. However, despite reports of high predictive accuracy, evidence supporting clinical implementation remains limited.
Objective:
This study aimed to evaluate the methodological rigor of AI-based ASCVD prediction models and assess whether reported algorithmic discrimination is associated with prespecified reporting and translational-evidence gaps.
Data Sources And Study Selection:
A systematic search of PubMed, Embase, Web of Science, Scopus, and Cochrane CENTRAL was conducted for original studies published between 1 January 2020 and 27 March 2026. Studies reporting the development or validation of AI and machine learning prognostic models for adult ASCVD outcomes were included.
Data Extraction And Synthesis:
Following the PRISMA guidelines, a structured methodological audit was performed. Five binary reporting and translational-evidence gaps were independently extracted: absent calibration assessment, lack of external validation, absent decision-analytic utility evaluation, undefined temporal prediction horizons, and lack of reproducibility or accessibility artifacts. A prespecified minimum evidence proxy for translational evaluation was defined.
Main Outcomes And Measures:
The primary outcomes were the cumulative methodological gap burden (range, 0-5) and its association with the primary discrimination metric (area under the receiver operating characteristic curve [AUC] or C-index).
Results:
Among 101 primary models evaluated, the median discrimination performance was 0.790 (IQR, 0.734-0.833). However, a substantial cumulative gap burden was observed, with only 6.9% of the models meeting the prespecified translational-readiness proxy. Reported discrimination was not associated with methodological completeness; top-quartile models (AUC ≥ 0.833) exhibited high gap burdens at rates comparable to lower-performing models. Reporting and translational-evidence gaps systematically co-occurred, predominantly driven by the simultaneous absence of external validation, calibration assessment, and reproducibility or accessibility artifacts. The use of advanced algorithmic architectures (e.g., deep learning) and multimodal data integration did not independently reduce the reporting gap burden.
Conclusion And Relevance:
In AI-based ASCVD risk prediction, reported algorithmic discrimination is a poor surrogate for clinical readiness. The literature features models with high discrimination but critical methodological deficits. These findings suggest that evaluation frameworks should consider incorporating calibration, external validation, and formal decision-analytic utility assessment alongside discrimination metrics.