Artificial intelligence in the GPs office: a retrospective study on diagnostic accuracy

Area of Science:

• Artificial Intelligence in Medicine
• Primary Health Care Diagnostics
• Machine Learning Applications

Background:

• Machine learning (ML) is poised to significantly impact primary health care (PHC).
• Limited peer-reviewed research exists on ML diagnostic accuracy versus general practitioners (GPs).
• Understanding ML performance and interpretability is crucial for clinical adoption.

Purpose of the Study:

• Evaluate the diagnostic accuracy of an ML classifier for primary headache diagnoses in PHC.
• Compare ML classifier performance against GPs.
• Identify key clinical features influencing ML diagnostic predictions.

Main Methods:

• Retrospective diagnostic accuracy study utilizing electronic health records from Iceland's PHCCA.
• Analysis included data from 15 PHC centers (2006-2020).
• Evaluated sensitivity, specificity, PPV, MCC, ROC, AUROC, and SHAP values.

Main Results:

• The ML classifier demonstrated superior performance over GPs across most diagnostic metrics, except specificity.
• Shapley Additive Explanations (SHAP) values confirmed the ML classifier utilizes clinically relevant features similar to physicians.
• The study provides quantitative evidence of ML's diagnostic capabilities in a real-world PHC setting.

Conclusions:

• The ML classifier exhibits superior diagnostic accuracy for primary headaches compared to GPs in a retrospective analysis.
• The ML classifier's reliance on physician-like features enhances its clinical interpretability and potential utility.
• This research highlights the potential of ML to augment diagnostic processes within primary health care.