Development, validation, and user-centric evaluation of an interpretable machine learning decision support tool for...

Mild bleeding disorders are the most common inherited bleeding disorders, often leading to perioperative haemorrhages. Preoperative screening for mild bleeding disorders remains challenging due to the limitations of existing screening tools, resulting in a substantial proportion of patients being referred for preoperative investigations. The aim of this study was to develop, externally validate, and implement an easy-to-use, explainable machine learning-based decision support tool for the prediction of mild bleeding disorders.

Clinical and laboratory data were collected in two independent, prospective cohort studies, including consecutive patients, aged 18 years or older, referred for suspected mild bleeding disorders. The training cohort was recruited at Inselspital, Bern University Hospital (Bern, Switzerland). Diagnostic investigations followed current guidelines, with final diagnoses established by an expert panel. Multiple machine learning algorithms were trained, and the best performing model underwent external validation in a second cohort recruited at Cantonal Hospital Lucerne (Lucerne, Switzerland). To evaluate usability, we created a survey platform incorporating four case vignettes and the System Usability Scale (SUS), a validated software usability questionnaire.

The training cohort included 555 patients (371 [67%] female and 184 [33%] male; median age 44 years [IQR 29-62]). The following predictors were selected: activated partial thromboplastin time, platelet function analysis with an epinephrine-collagen cartridge, sex, and a streamlined bleeding history. A focus group of relevant stakeholders first identified candidate variables reasonably available at pre-anaesthesia evaluation; final predictors were then selected using the Boruta algorithm in R. In the external validation cohort (n=217), 90·2% (95% CI 83·1-94·9) of patients with mild bleeding disorders were correctly predicted (sensitivity) and 54·3% (95% CI 44·3-64·0) of patients without mild bleeding disorders were correctly classified as not having mild bleeding disorders (specificity). The area under the receiver operating characteristic curve was 0·85 (95% CI 0·80-0·90). The final decision support tool was assessed by 33 surgeons, 29 anaesthesiologists, and 24 haematologists. The median time to complete the tool was 72 s (IQR 49·0-79·5). The median SUS score was 82·5 (IQR 72·5-90·0), indicating excellent usability.

MBD-Check is an interpretable machine learning solution that could simplify the preoperative prediction of mild bleeding disorders, potentially supporting more efficient referral decisions.

Swiss National Science Foundation.