Quality assurance in machine Learning: Do we hit the mark for total joint arthroplasty models? A systematic review

Machine learning (ML) models are increasingly used in total hip arthroplasty (THA) and total knee arthroplasty (TKA) to enhance outcomes and optimize clinical decision-making. However, quality assurance (QA) measures vary within models and are often inadequately reported. This study aims to provide a high-level overview of QA practices in ML models within THA and TKA.

A systematic review of published articles from 2013 to 2024 was conducted for the concepts of THA or TKA, ML, and QA. Two independent reviewers screened articles based on predefined criteria and extracted 31 features to measure QA performance. TRIPOD-AI and PROBAST guidelines were used to assess transparency of reporting and bias.

A total of 2428 articles were screened, with 137 studies included. Of these, 76.64 % were clinical outcome models, 22.63 % were imaging models, and 0.01 % were large language models. Validation methods were reported in 98.54 % of studies, AUC-ROC scores in 68.61 %, and overall accuracy in 53.28 %. However, recall and precision metrics were significantly underreported, with 94.89 % and 85.85 % of studies failing to report these metrics, respectively. The TRIPOD-AI analysis revealed a mean compliance score of 0.84 (0.60-1.00; SD 0.07). The PROBAST-AI risk of bias analysis showed an overall high degree of bias, specifically within the participants and analysis domains.

There is significant heterogeneity in reporting of QA metrics in ML studies for THA and TKA. As AI studies grow at an exponential rate, it is necessary that standardized QA guidelines are implemented to ensure reliability and validity of ML models.