Evaluation of Combined Artificial Intelligence and Radiologist Assessment to Interpret Screening Mammograms

Area of Science:

• Radiology
• Medical Imaging
• Artificial Intelligence

Background:

• Mammography screening interpretation is subjective and prone to human error.
• Artificial intelligence (AI) offers potential to improve accuracy by reducing missed cancers and false positives.

Purpose of the Study:

• To rigorously evaluate machine learning algorithms for interpreting screening mammograms.
• To determine if AI can overcome limitations in human mammography interpretation.

Main Methods:

• An international, crowdsourced challenge hosted over 1100 participants to develop AI algorithms for mammography interpretation.
• Algorithms were evaluated using images alone or combined with clinical data, with accuracy measured by area under the curve (AUC) and specificity at fixed radiologist sensitivity.
• An ensemble method combining top AI algorithms and radiologist assessments was developed.

Main Results:

• No single AI algorithm outperformed radiologists, but the top algorithm achieved AUCs of 0.858 (US) and 0.903 (Sweden).
• AI specificity was lower than community radiologists (66.2% US, 81.2% Sweden vs. 90.5% US, 98.5% Sweden).
• Combining top AI algorithms with US radiologist assessment yielded a higher AUC (0.942) and improved specificity (92.0%) at the same sensitivity.

Conclusions:

• While individual AI algorithms did not surpass radiologists, an ensemble approach with AI and radiologist collaboration improved overall mammography screening accuracy.
• This study highlights the potential of machine learning in enhancing mammography interpretation and patient outcomes.