Algorithm Versus Expert: Machine Learning Versus Surgeon-Predicted Symptom Improvement After Carpal Tunnel Release.

Area of Science:

• Orthopedic Surgery
• Medical Decision Making
• Biostatistics

Background:

• Surgeons' clinical experience guides treatment effect predictions.
• Algorithm-based predictions can support surgical decision-making.
• To add value, predictive models must surpass surgeon judgment.

Purpose of the Study:

• Compare a validated prediction model's performance against surgeon predictions for symptom improvement after carpal tunnel release (CTR).
• Evaluate the clinical utility and decision-making benefits of the prediction model versus surgeon estimates.

Main Methods:

• A cohort of 97 patients scheduled for CTR was studied.
• Surgeons estimated preoperative probabilities of achieving minimally clinically important difference in symptoms at 6 months post-surgery.
• Model and surgeon performance were assessed using calibration, discrimination (AUC), accuracy, sensitivity, and specificity.

Main Results:

• The prediction model demonstrated superior calibration and discrimination (AUC 0.77) compared to surgeon predictions (AUC 0.62).
• The model achieved higher accuracy (0.78 vs 0.65) and sensitivity (0.85 vs 0.72) than surgeons.
• Net benefit analysis favored the prediction model, indicating improved decision-making with more correctly predicted improvements.

Conclusions:

• The prediction model significantly outperformed surgeon predictions in calibration, accuracy, and sensitivity for post-CTR symptom improvement.
• Utilizing the prediction model enhances patient selection for CTR, maximizing improvements while avoiding unnecessary surgeries.