Experiences with a self-test for Dutch breast screening radiologists: lessons learnt

Area of Science:

• Radiology
• Medical Imaging
• Quality Assurance in Healthcare

Background:

• National quality assurance programs are crucial for maintaining high standards in medical imaging.
• Breast screening programs rely on accurate interpretation of mammograms by radiologists.
• Evaluating radiologist performance is essential for continuous improvement in diagnostic accuracy.

Purpose of the Study:

• To assess the effectiveness of a self-test designed for Dutch breast screening radiologists.
• To evaluate radiologist performance within the national quality assurance framework.
• To identify areas for improvement in mammogram interpretation and reporting.

Main Methods:

• 144 radiologists participated in a self-assessment using 60 screening mammograms (20 with malignancies).
• Participants identified lesion location, type, and assigned BI-RADS categories.
• Performance metrics included areas under the receiver operating characteristics (ROC) curves (AUC), sensitivity, specificity, and interobserver agreement (kappa statistics).

Main Results:

• 110 radiologists (76%) completed the test, reading a median of 10,000 mammograms annually.
• Median AUC was 0.93, with case and lesion sensitivity at 91% and case specificity at 94%.
• Substantial agreement was found for recall (κ=0.77) and laterality (κ=0.80), while moderate agreement was observed for lesion type (κ=0.57) and BI-RADS (κ=0.45).

Conclusions:

• The self-test demonstrated satisfactory performance regarding ROC curves, sensitivity, and recall agreement.
• Areas needing improvement include agreement on lesion type and BI-RADS classification, indicating a need for targeted education.
• Individual and group feedback were provided, with future research planned to assess performance improvements.