Integrating conformal prediction with machine learning for uncertainty-aware risk stratification in coronary artery
Shengnan Lin1, Xuyang Hu1, Yang Zhang2
1Department of Cardiology, Liaoning University of Traditional Chinese Medicine, Shenyang, Liaoning, China.
Background:
Predictive models are increasingly used in clinical decision-making in coronary artery disease (CAD). However, most existing models focus on discriminative ability while ignoring individual prediction uncertainty, which is particularly prominent in small-sample contexts, limiting clinical applications.
Methods:
We developed an uncertainty-aware model integrating machine learning and conformal prediction (CP) for CAD prediction in a small-sample setting cohort (N = 297). Logistic regression (LR) and random forest (RF) models were trained using clinical variables and evaluated for predictive performance. CP was applied to quantify prediction uncertainty and generate prediction sets with statistical coverage guarantees. Model reliability was assessed using coverage and prediction set characteristics. By combining predicted risk probabilities with uncertainty information, the model transforms traditional binary classification into a four-class framework that reflects both the level of risk (high vs. low) and the certainty of the prediction (high vs. low), thereby enabling a more informative risk stratification.
Results:
Both LR and RF show strong discrimination (AUC 0.953 and 0.951). LR is slightly better calibrated, while RF is more accurate. CP efficiently quantifies uncertainty. Union and intersection achieve higher coverage, whereas MSCP with voting balances coverage and efficiency. The combined RF-MSCP model maintains empirical coverage above the nominal level across all significance thresholds, demonstrating robust uncertainty assessment and risk prediction in small-sample settings.
Conclusion:
The integration of CP with machine learning has led to the development of the RF-MSCP model, which provides reliable uncertainty quantification for CAD risk prediction, especially in small sample settings. By identifying potentially high-risk individuals overlooked by traditional models, this approach improves predictive discrimination and supports more robust clinical decision.
