Transparency in Externally Validated Models: A systematic review of machine learning vs. logistic regression for
Sara Ben Hmido1, Houssam Abder Rahim2, Erik W Ingwersen1
1Department of Surgery, Amsterdam University Medical Centers, De Boelelaan 1117, 1081, HV Amsterdam, the Netherlands; Cancer Center Amsterdam, De Boelelaan 1118, 1081, HV Amsterdam, the Netherlands.
Introduction:
Colorectal resection carries a 2.8 %-30 % risk of anastomotic leakage. Machine learning can estimate risks and guide decisions, but clinical implementation remains inadequate due to transparency issues. This review assesses the performance and transparency of machine learning models compared to logistic regression.
Methods:
A systematic review followed PRISMA guidelines. Medline, Embase, Web of Science, and Cochrane databases were searched for studies using Logistic Regression or Machine Learning with external validation for colorectal anastomotic leakage prediction. Data were extracted using CHARMS, risk of bias assessed with PROBAST, and transparency with TRIPOD + AI.
Results:
Ten studies were included. Machine learning models were validated on smaller cohorts than logistic regression. Transparency scores ranged from 29 % to 63 %, averaging 45 % for logistic regression and 43 % for machine learning. Reporting of missing data was inconsistent, and external validation was limited. Most studies had a high risk of bias due to small sample sizes and low event counts.
Conclusion:
In comparison to Logistic regression studies, machine learning studies are limited by small cohorts, low outcome numbers, and a lower level of transparency. Future research should prioritise transparency, adhere to TRIPOD + AI standards, and develop LR and ML models in parallel using the same datasets while ensuring separate models for colon and rectal surgery. Currently, these models are not yet suitable for clinical implementation; more robust and transparent models must be developed based on these recommendations before they can be applied in clinical practice.
