Assessing the predictive capacity of machine learning models using patient-specific variables in determining
Fong H Nham1, Tannor Court1, Abdul K Zalikha1
1Department of Orthopaedic Surgery and Sports Medicine, Detroit Medical Center, Detroit, MI, 48201, USA.
Background:
Machine learning is a subset of artificial intelligence using algorithmic modeling to progressively learn and create predictive models. Clinical application of machine learning can aid physicians through identification of risk factors and implications of predicted patient outcomes.
Aims:
The aim of this study was to compare patient-specific and situation perioperative variables through optimized machine learning models to predict postoperative outcomes.
Methods:
Data from 2016 to 2017 from the National Inpatient Sample was used to identify 177,442 discharges undergoing primary total hip arthroplasty, which were included in the training, testing, and validation of 10 machine learning models. 15 predictive variables consisting of 8 patient-specific and 7 situational specific variables were utilized to predict 3 outcome variables: length of stay, discharge, and mortality. The machine learning models were assessed in responsiveness via area under the curve and reliability.
Results:
For all outcomes, Linear Support Vector Machine had the highest responsiveness among all models when using all variables. When utilizing patient-specific variables only, responsiveness of the top 3 models ranged between 0.639 and 0.717 for length of stay, 0.703-0.786 for discharge disposition, and 0.887-0.952 for mortality. The top 3 models utilizing situational variables only produced responsiveness of 0.552-0.589 for length of stay, 0.543-0.574 for discharge disposition, and 0.469-0.536 for mortality.
Conclusions:
Linear Support Vector Machine was the most responsive machine learning model of the 10 algorithms trained, while decision list was most reliable. Responsiveness was observed to be consistently higher with patient-specific variables than situational variables, emphasizing the predictive capacity and value of patient-specific variables. The current practice in machine learning literature generally deploys a single model, it is suboptimal to develop optimized models for application into clinical practice. The limitation of other algorithms may prohibit potential more reliable and responsive models.Level of Evidence III.
